perf: add missing indexes and remove N+1 queries

The app degraded progressively over a year of data growth. Four causes,
all cumulative:

1. No indexes. Apart from primary keys, unique constraints and
   session.expire, no table carried an index. PostgreSQL does not index
   foreign keys automatically, so every filter and join on group_id,
   supplier_id, order_id and the date columns did a sequential scan.
   Worst offender: user_groups.user_id, read by getUserWithGroups() on
   every authenticated request.

2. N+1 in the order and delivery listings. getOrders,
   getOrdersByDateRange, getDeliveries and getDeliveriesByDateRange
   issued one to two queries per row to load relations. Relations are now
   loaded in bulk and grouped in Node: three queries regardless of volume.

3. /api/sync-order-delivery-status reloaded the whole deliveries table on
   every iteration of its loop over orders. It now uses the deliveries
   getOrders() already attaches.

4. clearExpiredCache() was implemented but never called, so
   invoice_verification_cache grew without bound. Now scheduled every 6h.

Also replaces the full-history downloads on the Groups and Suppliers
pages, which fetched every order and delivery with nested relations only
to count rows, with aggregate endpoints that count in the database.

Indexes are created via scripts/auto-migrate-production.sh, the script
that actually runs at deploy time, using CREATE INDEX CONCURRENTLY so no
write lock is taken. Note that server/migrations.ts explicitly ignores
the migrations/ directory and runs only hardcoded migrations; the SQL
file added there is for reference and manual application.

Verified: typecheck baseline 440 errors, 430 after, none new in the
changed code; vite build passes; server boots; functional test confirms
the aggregate endpoints match the source data including the delivered
count.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
MichaelandClaude Fable 5 committed 2026-08-14 11:55:49 +02:00
1 parent 693d9460cb
commit 753b301066
9 files changed
+678 -290

No files matched your search

+12 -41
View File
@@ -18,7 +18,7 @@ import {
Truck,
Palette
} from "lucide-react";
import type { Group, NocodbConfig } from "@shared/schema";
import type { EntityActivityStats, Group, NocodbConfig } from "@shared/schema";
const colorOptions = [
{ value: '#1976D2', label: 'Bleu' },
@@ -77,12 +77,10 @@ export default function Groups() {
environment: window.location.hostname
});
const { data: orders = [] } = useQuery({
queryKey: ['/api/orders'],
});
const { data: deliveries = [] } = useQuery({
queryKey: ['/api/deliveries'],
// Comptages agrégés en base : la page n'affiche que des totaux, inutile de
// rapatrier l'historique complet des commandes et des livraisons.
const { data: groupStats = [] } = useQuery<EntityActivityStats[]>({
queryKey: ['/api/stats/by-group'],
});
const createMutation = useMutation({
@@ -214,42 +212,15 @@ export default function Groups() {
group.name.toLowerCase().includes(searchTerm.toLowerCase())
) : [];
console.log('🏪 Groups Debug:', {
groups: Array.isArray(groups) ? groups.length : 'NOT_ARRAY',
orders: Array.isArray(orders) ? orders.length : 'NOT_ARRAY',
deliveries: Array.isArray(deliveries) ? deliveries.length : 'NOT_ARRAY',
groupsData: groups?.slice(0, 2),
ordersData: orders?.slice(0, 2),
deliveriesData: deliveries?.slice(0, 2)
});
const getGroupStats = (groupId: number) => {
const groupOrders = Array.isArray(orders) ? orders.filter(order => {
const match = order.groupId === groupId;
if (match) console.log(`🎯 Order ${order.id} matches group ${groupId}`, order);
return match;
}) : [];
const groupDeliveries = Array.isArray(deliveries) ? deliveries.filter(delivery => {
const match = delivery.groupId === groupId;
if (match) console.log(`🎯 Delivery ${delivery.id} matches group ${groupId}`, delivery);
return match;
}) : [];
console.log(`🏪 Group ${groupId} stats:`, {
groupOrders: groupOrders.length,
groupDeliveries: groupDeliveries.length,
delivered: groupDeliveries.filter(d => d.status === 'delivered').length,
ordersSample: groupOrders.slice(0, 2),
deliveriesSample: groupDeliveries.slice(0, 2),
allOrdersCount: orders?.length,
allDeliveriesCount: deliveries?.length
});
const stats = Array.isArray(groupStats)
? groupStats.find(s => s.id === groupId)
: undefined;
return {
orders: groupOrders.length,
deliveries: groupDeliveries.length,
delivered: groupDeliveries.filter(d => d.status === 'delivered').length,
orders: stats?.orders ?? 0,
deliveries: stats?.deliveries ?? 0,
delivered: stats?.delivered ?? 0,
};
};
+12 -14
View File
@@ -29,7 +29,7 @@ import {
CheckCircle,
Clock
} from "lucide-react";
import type { Supplier } from "@shared/schema";
import type { EntityActivityStats, Supplier } from "@shared/schema";
export default function Suppliers() {
const { user } = useAuthUnified();
@@ -56,12 +56,10 @@ export default function Suppliers() {
queryKey: ['/api/suppliers'],
});
const { data: orders = [] } = useQuery({
queryKey: ['/api/orders'],
});
const { data: deliveries = [] } = useQuery({
queryKey: ['/api/deliveries'],
// Comptages agrégés en base : la page n'affiche que des totaux, inutile de
// rapatrier l'historique complet des commandes et des livraisons.
const { data: supplierStats = [] } = useQuery<EntityActivityStats[]>({
queryKey: ['/api/stats/by-supplier'],
});
const createMutation = useMutation({
@@ -280,14 +278,14 @@ export default function Suppliers() {
);
const getSupplierStats = (supplierId: number) => {
const supplierOrders = orders.filter((order: any) => order.supplierId === supplierId);
const supplierDeliveries = deliveries.filter((delivery: any) => delivery.supplierId === supplierId);
const deliveredCount = supplierDeliveries.filter((delivery: any) => delivery.status === 'delivered').length;
const stats = Array.isArray(supplierStats)
? supplierStats.find(s => s.id === supplierId)
: undefined;
return {
orders: supplierOrders.length,
deliveries: supplierDeliveries.length,
delivered: deliveredCount,
orders: stats?.orders ?? 0,
deliveries: stats?.deliveries ?? 0,
delivered: stats?.delivered ?? 0,
};
};