Michael/CollectFlow
1
0 Fork 0
mirror of https://github.com/R0m1k3/CollectFlow.git synced 2026-10-11 17:26:32 +02:00
Code Issues Packages Projects Releases Wiki Activity
446 Commits 16 Branches 0 Tags
8bd45518f2c543cc15415e77c87df0872d1caf8e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MichaelandClaude Opus 4.8 8bd45518f2 chore(qlik): log raw sample of 4 new metrics to isolate empty columns 
Adds a sample log of the first extracted rows (all 8 hypercube columns)
and a sample of the network metrics matched from cache in getProductRows,
to pinpoint whether the new measures return null from Qlik, fail to
persist, or fail to render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 08:39:35 +02:00
MichaelandClaude Opus 4.8 bd9bc78f4c feat(qlik): extract 4 decision metrics (CA/mag, couverture, marge%, rupture%) 
Adds CA par Magasin, Couverture stock Qte, Marge % and Stock Rupture %
network measures to the Qlik hypercube extraction (in-page Playwright ws
+ legacy server ws), persists them in qlik_network_metrics, joins them in
getProductRows and renders 4 new grid columns with buy-signal coloring
(low coverage / high margin = green; high rupture = rose).

- qlik-client: config IDs (env-overridable, master GUIDs of app 9872ee6e),
  NetworkMetric fields, hypercube width 4->8, page height 2500->1250
- qlik-playwright: same 4 measures in the in-page hypercube + row mapping
- schema + db-init: 4 numeric columns (ALTER ADD COLUMN IF NOT EXISTS for
  existing DBs)
- cache + grid types + enrich: plumb the new fields end to end

Note: percent measures stored as raw Qlik ratio; UI renders x100 when |v|<=1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 08:34:54 +02:00
MichaelandClaude Opus 4.8 0b9846f55d fix(qlik): auto-refresh grid after sync without manual reload 
Restore the _refresh searchParam bump so grid-client refetches
/api/grid/rows with refresh=1 (forceRefresh, bypassing the 10-min
getProductRows cache) and re-renders the network columns from the
store. Also call router.refresh() to update the server-rendered
MAJ Qlik timestamp on the button.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 08:23:34 +02:00
MichaelandClaude Opus 4.8 d54c3ae520 feat(qlik): add step-by-step logging to sync extraction 
Logs each stage (PG codes -> NTLM session -> chromium -> csrf token
-> in-page ws -> OpenDoc -> SelectValues -> cube size -> rows) so a
hanging sync reveals where it stalls. Forwards in-page browser console
to the node logger.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 08:14:12 +02:00
MichaelandClaude Opus 4.8 6cdb845a1f fix(qlik): use qlik-csrf-token + real engine qIds — extraction works 
Validated end-to-end. The recent Qlik client authorizes the engine websocket
via ?reloadUri=...&qlik-csrf-token=<token> (not Xrfkey). Capture that token
from the client's own ws, then open ours in-page with it. Use the real engine
master-item qIds (dim Article Code=yesCP, CA N=JhqJ, Quantité N=41516861...,
Magasin Ventes Nb N=yNBLjc) — the QRS object IDs returned error 7001. Select
the supplier's codes on field "Article Code" (the app holds ~1.2M codes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 00:37:04 +02:00
MichaelandClaude Opus 4.8 b755a69258 fix(qlik): in-page engine ws with authenticated browser session 
The recent Qlik client uses qmfe/single-spa (no js/qlik Capability API). Go
back to a raw engine websocket but opened in-page from the authenticated
browser (injected X-Qlik-Session cookie), which the proxy accepts. Use the
session xrfkey and capture ws close code/reason for diagnostics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 00:19:28 +02:00
MichaelandClaude Opus 4.8 d319107076 fix(qlik): extract via Qlik Capability API (reuse page session) 
Raw and in-page websockets with a self-generated xrfkey are rejected (403).
Switch to the Qlik Capability API (require(['js/qlik']) → openApp → createCube),
which reuses the authenticated page session and manages the engine ws + xrfkey
internally. Auth via injected X-Qlik-Session cookie (Node ticket flow) +
httpCredentials. Logs rich page diagnostics on failure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-21 00:13:45 +02:00
MichaelandClaude Opus 4.8 694b780160 feat(qlik): extract network data via Playwright (in-page ws) 
The Qlik proxy refuses a raw server-side websocket (403, even internally), so
extraction now runs through headless Chromium (NTLM via httpCredentials, like
the hermes agent) and opens the Engine websocket in-page — same origin, which
the proxy accepts. Efficient: selects the supplier's article codes on the
"Article Code" field so the hypercube returns only those rows.

- qlik-playwright.ts: browser singleton, in-page hypercube extraction (paginated)
- /api/qlik/sync: use the Playwright extractor
- Dockerfile: install chromium + headless deps, PLAYWRIGHT_CHROMIUM_PATH, copy playwright-core
- deps: playwright-core (lockfiles synced)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 23:54:03 +02:00
MichaelandClaude Opus 4.8 1281b0e21f fix(qlik): send Origin/User-Agent on engine ws + verbose 403 diagnostics 
Qlik proxy validates the WebSocket Origin header (whitelist) — a browser
always sends it. Add Origin + User-Agent to the engine ws. Also surface the
403 response (server/www-authenticate/body) for diagnosis.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 23:35:09 +02:00
MichaelandClaude Opus 4.8 abed369bfb feat(settings): configure Qlik credentials in the UI 
Add a "Qlik Sense — Données réseau" section in Paramètres (host/user/password
+ Save & Test buttons), stored in data/.db-config.json. qlik-client reads this
file first (env fallback), so no env vars needed for the sync.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 23:28:55 +02:00
MichaelandClaude Opus 4.8 2ffa7dd1c0 build: sync package-lock.json with httpntlm/ws deps 
Docker build uses `npm ci` which requires package-lock.json in sync with
package.json. Add httpntlm, ws, @types/ws to the npm lockfile (were only in
pnpm-lock.yaml), fixing the failed deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 23:19:27 +02:00
MichaelandClaude Opus 4.8 aa05606b54 build: sync package-lock.json with httpntlm/ws deps 
Docker build uses `npm ci` which requires package-lock.json in sync with
package.json. Add httpntlm, ws, @types/ws to the npm lockfile (were only in
pnpm-lock.yaml), fixing the failed deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 22:55:18 +02:00
MichaelandClaude Opus 4.8 aee968d30f feat(grid): replace ranking/AI/score with Qlik network data 
Remove ranking, AI analysis (routes + ai-copilot module), and score-engine
from the grid. Add Qlik Sense network metrics (CA, Qte, nb magasins per
product) joined by code centrale (articles.artcentrale).

- qlik-client.ts: NTLM + ticket SSO auth, QIX hypercube extraction (master items)
- qlik-network-cache.ts + qlik_network_metrics table (db-init): cache layer
- POST /api/qlik/sync?fournisseur=: per-supplier sync into cache
- SyncQlikButton: per-supplier sync, grid reload, last-update date display
- get-product-rows Phase 8: enrich rows with cached network metrics
- heatmap-grid: CA reseau / Qte reseau / Magasins (/270) / % presence columns
- discovery: app 9872ee6e, master items CA N / Quantite N / Magasin Ventes Nb N
- deps: httpntlm, ws

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-20 22:52:39 +02:00
Michael fbad53df36 feat(ai-copilot): add AI analysis engine with prompt generation, recommendation extraction, and insight cleaning capabilities. 2026-02-23 22:52:21 +01:00
Michael 17498dcc10 feat: Add bulk AI analyzer component to process product data and generate insights. 2026-02-23 00:40:49 +01:00
Michael 8c02771e46 feat: Introduce AI Copilot for bulk product analysis, including a dedicated store, UI components, and business logic. 2026-02-23 00:29:45 +01:00
Michael 03c5ab6d9e feat: Add bulk AI product analysis functionality with data models, state management, and a UI component. 2026-02-23 00:14:22 +01:00
Michael 673cab4454 feat: Add bulk AI product analysis component with progress tracking, concurrency, and benchmark calculations, along with a component for displaying individual AI insights. 2026-02-22 23:58:19 +01:00
Michael 960045dc53 feat: add OpenRouter client for AI product analysis. 2026-02-22 23:54:44 +01:00
Michael b92ed905d7 feat: Implement AI copilot product analysis logic and OpenRouter integration. 2026-02-22 18:49:09 +01:00
Michael 90c38bca8b feat: Implement a new Sidebar component featuring dynamic navigation links, user session display, and logout functionality. 2026-02-22 18:38:42 +01:00
Michael 2fbe88be71 feat: Add a new Sidebar component for application navigation, user profile display, and logout functionality. 2026-02-22 18:27:51 +01:00
Michael 43c625c3ec feat: Introduce SnapshotList component for managing user snapshots and exports, leveraging a new ConfirmModal. 2026-02-22 14:43:48 +01:00
Michael bab862bf5c feat: Add server-side function getSnapshots to retrieve session snapshots from the database. 2026-02-22 14:39:42 +01:00
Michael d6e313f945 feat: Implement a snapshots feature with a list component and a dedicated page to manage and restore saved sessions. 2026-02-22 14:37:19 +01:00
Michael afa7e59c34 feat: Implement Apple-inspired design system with global dark/light mode styles and utility classes. 2026-02-22 14:32:42 +01:00
Michael 997abee21a feat: Introduce an admin user management component to create, delete, and update user passwords. 2026-02-22 14:26:16 +01:00
Michael 44d7f95d4c feat: implement user authentication and route protection using NextAuth middleware and server actions. 2026-02-22 14:23:11 +01:00
Michael a03fd64b5d feat: Implement NextAuth for credentials-based authentication and add initial admin user seeding logic. 2026-02-22 14:20:31 +01:00
Michael cfd079fbf5 feat: add settings page for database, OpenRouter, score, theme, and display density configurations. 2026-02-22 14:16:49 +01:00
Michael 359d196c4d feat: implement user authentication and authorization with NextAuth, including user schema, password hashing, and role-based access control. 2026-02-22 14:11:36 +01:00
Michael c842c2447a feat: Implement snapshot and export listing, restoration, and deletion with a new success modal and floating summary bar. 2026-02-22 13:54:24 +01:00
Michael a93b1b36fc feat: Implement snapshot and export management, including API endpoints, database schema, and UI for listing, saving, loading, and deleting. 2026-02-22 13:49:51 +01:00
Michael 7288b3da72 feat: implement server-side function to save session snapshots, including input validation and database table creation fallback. 2026-02-22 13:41:49 +01:00
Michael 0bec6b953b feat: implement server action to save session snapshots, including table auto-creation. 2026-02-22 13:39:01 +01:00
Michael 3b3021d85f feat: Add API to save session snapshots and initialize the corresponding database table. 2026-02-22 13:35:34 +01:00
Michael 77fb61ef79 feat: Add floating summary bar with draft management, snapshot, and Excel export functionalities. 2026-02-22 13:31:38 +01:00
Michael c9cf83d4c7 feat: implement session snapshot creation and history viewing with new pages and components. 2026-02-22 13:29:12 +01:00
Michael 7765f97a1f feat: Add AI Copilot store for product analysis and a floating summary bar component with AI insight integration. 2026-02-22 13:24:59 +01:00
Michael 804328a525 feat: Implement a new heatmap grid component for product data visualization and selection, and create new bulk AI analyzer and floating summary bar components. 2026-02-22 13:23:11 +01:00
Michael a04d6d4e7a feat: add product score explanation page detailing calculation, principles, and interpretation. 2026-02-22 13:18:19 +01:00
Michael 8767b1cf1e feat: add AI Copilot Zustand store for managing product insights and analysis. 2026-02-22 13:12:34 +01:00
Michael c23f74d32d feat: Add Excel export for modified gammes via a new API route and a button in the floating summary bar. 2026-02-22 13:10:10 +01:00
Michael b0d25eb4d7 feat: Add FloatingSummaryBar component to display grid summary, draft changes, and save/export actions. 2026-02-22 13:05:15 +01:00
Michael 8b9803fffc feat: introduce AI-powered bulk product analysis feature with dedicated engine, types, and UI component. 2026-02-22 12:32:09 +01:00
Michael 335cf15ffa feat: Introduce AI copilot for bulk product analysis and integrate a new product scoring engine. 2026-02-22 12:28:50 +01:00
Michael 425d9e8088 feat: Add Bulk AI Analyzer component to process products, call an AI analysis API with concurrency and retry logic, and update grid insights and recommendations. 2026-02-22 12:21:05 +01:00
Michael 661704bf38 feat: Add bulk AI product analysis component with supporting data types and AI system prompt. 2026-02-22 12:11:32 +01:00
Michael b015d1f093 feat: Add AnalysisEngine for generating AI prompts and extracting recommendations for product range analysis. 2026-02-22 12:08:35 +01:00
Michael 8e2c2d4c15 feat: Add AnalysisEngine for AI copilot product analysis prompt generation and recommendation extraction. 2026-02-22 12:02:42 +01:00
Michael 3de16d15d7 feat: Introduce bulk AI product analysis, integrating with OpenRouter via a new API and UI. 2026-02-22 11:59:15 +01:00
Michael 04e6ed70f6 feat: Implement bulk AI product analysis functionality with a new UI component and API route. 2026-02-22 11:19:54 +01:00
Michael 7f9c07b58e feat: Add bulk AI product analysis feature with a new UI component and API route. 2026-02-22 11:00:44 +01:00
Michael 4b3672e65a feat: Add AI product analysis endpoint supporting individual and batch processing modes. 2026-02-22 09:42:51 +01:00
Michael 9744502788 feat: add AI product analysis API route supporting individual and batch modes 2026-02-22 09:23:04 +01:00
Michael ce0085a5c7 feat: Implement bulk AI product categorization through a new UI component and API route. 2026-02-22 09:17:52 +01:00
Michael 9c79b631cb feat: implement batch AI product analysis API route and UI component for product categorization. 2026-02-22 09:10:19 +01:00
Michael d644043aa5 feat: Add batch AI product categorization API and a bulk analyzer UI component. 2026-02-22 01:07:17 +01:00
Michael 259fb60638 feat(ai): performance-first logic with store awareness extrapolation and change count fix 2026-02-22 00:21:53 +01:00
Michael c5cacd9b5a fix(ai): final overhaul to performance-first, supplier-agnostic logic 2026-02-22 00:12:14 +01:00
Michael 0bb73ebcca feat(ai): update batch analysis to use global supplier weight (CA/Margin) 2026-02-21 23:27:54 +01:00
Michael c89e046dee feat(ai): synchronize analysis quality, optimize batch performance and implement context-aware weighting 2026-02-21 23:14:12 +01:00
Michael e8cc89cb2f feat(ai): synchronize batch and individual analysis payloads and models, optimize concurrency 2026-02-21 22:48:16 +01:00
Michael 66f28956ce fix(ai): fix syntax error in batch prompt and update types for score support 2026-02-21 22:41:48 +01:00
Michael 5c8910825b feat(ai): integrate performance score for smarter analysis and parallelize batch processing 2026-02-21 22:41:10 +01:00
Michael 4dac21dfac fix(grid): contextualize draft changes count and validation to visible products 2026-02-21 22:39:00 +01:00
Michael fe8d0f1b32 fix(ai): improved seasonality detection in batch by sending detailed month-by-month history 2026-02-21 22:28:41 +01:00
Michael 4f7701fa14 fix(ai): simplified categories to A (Permanent), C (Seasonal), Z (Exit) and removed B 2026-02-21 22:24:58 +01:00
Michael a186bb1989 fix(ai): prevent AI from using placeholder justification string in batch analysis 2026-02-21 22:20:07 +01:00
Michael 1ec8ae3f02 fix(ai): aligned batch system prompt with successful individual analysis prompt 2026-02-21 22:17:21 +01:00
Michael 816ec6e483 feat(ai): smarter batch analysis with revenue, history, nomenclature and strict retail rules 2026-02-21 22:09:28 +01:00
Michael 8ad2894301 fix(ai): batch-analyze uses user-configured model (Gemini 2.5) instead of hardcoded Llama 3.3 2026-02-21 22:05:18 +01:00
Michael 3d26dc0b9b fix(ai): client-side retry on 429, server returns retryAfter, hardcoded llama-3.3 free model 2026-02-21 22:04:08 +01:00
Michael 41c3ab34b6 fix(ai): server-side retry-with-backoff on 429, reads Retry-After header, up to 3 attempts 2026-02-21 21:58:25 +01:00
Michael 163d80dfe2 fix(ai): throttle now applies on ALL chunk iterations (incl. errors), delay bumped to 20s 2026-02-21 21:56:16 +01:00
Michael bcbea69d0a fix(ai): add 3s throttle between batch chunks to avoid OpenRouter 429 rate limiting 2026-02-21 21:53:14 +01:00
Michael d3dfcf6be4 fix(ai): batch-analyze now reads OpenRouter key from DB config instead of env var 2026-02-21 21:50:24 +01:00
Michael 1cda9cef9d debug(ai): add verbose logging and improved error visibility in batch analyzer 2026-02-21 21:45:58 +01:00
Michael c6148b58a4 fix(ai): prefix insight text with recommended gamme letter for clarity 2026-02-21 21:39:52 +01:00
Michael 09f9c0b097 feat(ai): bulk analysis now populates AI insight column with justification and duplicate flag 2026-02-21 21:38:27 +01:00
Michael a07bad60b2 feat(ai): integrate Llama 3.3 batch analysis for products grouped by nomenclature 2026-02-21 20:27:08 +01:00
Michael 09e00fbcdd fix(ui): resolve stale closure in excel export button preventing export of actual drafts 2026-02-21 20:08:35 +01:00
Michael 515ca55836 feat(export): export modified gammes to excel with specified columns 2026-02-21 20:04:46 +01:00
Michael 90205bf51b fix(ui): prevent icon overlap in nomenclature select 2026-02-21 19:08:45 +01:00
Michael d3b349cc31 feat(grid): make gamme filters dynamic based on available data and support Aucune state 2026-02-21 19:02:44 +01:00
Michael 86c6946c3e style(ux): final refinements to inline selection and visual balance 2026-02-21 18:53:22 +01:00
Michael c2dbaa7fbc style(ux): harmonize ui according to ergonomics specification (silent ui, tabular nums, muted icons) 2026-02-21 18:52:11 +01:00
Michael 9ef846a5d9 feat(grid): display dynamic reference counts per gamme in filter bar 2026-02-21 18:48:11 +01:00
Michael a94489dd07 fix(grid): preserve active grid query in store to restore context on navigation 2026-02-21 18:42:00 +01:00
Michael 9493a39d12 feat(grid): preserve initial gamme reference and add modification icon 2026-02-21 18:30:48 +01:00
Michael a0f8d9c373 fix(grid): adjust nomenclature filter padding to prevent icon overlap 2026-02-21 18:26:12 +01:00
Michael 8567361107 feat(grid): refine nomenclature to 6-digits, single dropdown filter, and distinct store colors 2026-02-21 18:07:36 +01:00
Michael 93e01a475a fix(db): remove non-existent columns from schema and fix aggregation logic 2026-02-21 17:34:04 +01:00
Michael 33615bb1bd feat(grid): implement hierarchical nomenclature filter (Secteur > Rayon > Famille) 2026-02-21 17:27:33 +01:00
Michael de7aa1560e feat(grid): improve draft persistence and fix global save in national view 2026-02-21 15:49:52 +01:00
Michael cb7cbf873f feat(grid): add store participation icons in designation column 2026-02-21 15:44:53 +01:00
Michael 29936cf3e1 feat(grid): split CA and Marge columns and enable sorting 2026-02-21 15:33:09 +01:00
Michael b9311503b2 ui(grid): simplify Gamme options to letters and center the column header 2026-02-21 15:16:15 +01:00
Michael 86240f4b46 chore(cleanup): remove unused imports, variables and fix missing hook dependencies 2026-02-21 15:15:22 +01:00
Michael c1bd0ae097 feat(grid): exclude current month from grid and scoring calculations 2026-02-21 15:13:29 +01:00