Default tts_engine changed from "edge" to "gemini".
Both /process-reel and /preview-tts now try Gemini first,
fall back to Edge TTS on failure.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Gemini returns raw PCM (audio/L16;codec=pcm;rate=24000), not a real WAV
file with a header, so FFmpeg fails to auto-detect the format. Pass
-f s16le, -ar (from mime type), and -ac 1 explicitly so FFmpeg can
decode the byte stream. Also surface FFmpeg stderr on failure instead
of swallowing it.
https://claude.ai/code/session_01QBvwHAMZzVYfWvy1U5paat
Replace the texttospeech.googleapis.com call (which requires a separate
GCP project with Cloud TTS enabled and billing) with the Gemini native
TTS endpoint (gemini-2.5-flash-preview-tts). This uses the same Gemini
API key already configured in the app, with no extra GCP setup needed.
Voice mapping: fr-FR-Standard-A/C -> Kore (female), B/D -> Charon (male).
Audio is returned as WAV and converted to MP3 via FFmpeg. Subtitle sync
uses ffsubsync as Gemini TTS does not return word boundaries.
https://claude.ai/code/session_01QBvwHAMZzVYfWvy1U5paat
Replace \uXXXX surrogate-pair escapes with the actual emoji characters
in print() calls. Lone surrogates are invalid in UTF-8, causing
UnicodeEncodeError on stdout encode and aborting TTS generation before
the Gemini API call.
https://claude.ai/code/session_01QBvwHAMZzVYfWvy1U5paat
- Add response structure logging to see actual keys returned by Gemini API
- Log timepoints count and first item to diagnose word boundary parsing
- Add 'unexpected timepoint format' warning for easier debugging
- Convert emoji to ASCII-safe versions for container logs
- Edge TTS fallback list now includes fr-FR-RemyMultilingualNeural
- Removed 'Neural not in voice' check that rejected Gemini voices (fr-FR-Standard-A etc) before falling back - these are valid voices and Edge-tts handles them properly
- Improved is_male check to match 'Remy' not 'Remi'
- Store Gemini API key globally in appConfig (single key for all users)
- Add /api/settings/gemini GET/POST/DELETE routes for API key management
- Backend: add tts_engine parameter ("edge" or "gemini") to FFmpeg service
- Backend: add generate_tts_gemini() using Google Cloud TTS REST API
- Frontend: Settings page shows Google Gemini API key input card
- Frontend: new-reel, mobile/new-reel, remotion-video, mobile/remotion-video
pages now have Edge/Gemini engine toggle and French voice selector
- Fix tts-preview route to extract ttsVoice from req.body instead of
undefined voice variable
- Remove piper_url from ReelRequest model and all function signatures
- Remove generate_tts_piper() function and Piper branch in generate_tts_with_subs()
- Remove /api/piper/config routes from server/routes.ts
- Remove piperConfig table, schemas and storage methods
- Remove piper_config migration
- Remove Piper TTS settings UI card
- Update "Piper TTS" labels to "TTS — voix activée"
edge_tts is now the only TTS engine, using precise word-boundary timing
Remove Minimax Speech API and Freesound integrations entirely.
Add Piper TTS as the sole TTS provider via configurable HTTP URL.
- Add piper_config DB table (url field, per-user)
- Add GET/POST /api/piper/config routes
- Python: replace generate_tts_minimax with generate_tts_piper (GET ?text=, WAV→MP3)
- Simplify generate_tts_with_subs: piper_url param replaces tts_provider+minimax fields
- Drop ttsVoice/ttsProvider from all UI, API, and background job params
- Settings page: replace Minimax+Freesound cards with single Piper URL input
- Remove freeSoundService init from server startup and CSP headers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pass GroupId query param to Minimax T2A v2 API — required for paid
plan quota allocation. Without it, Minimax defaults to (0/0 used).
- shared/schema.ts: groupId field on minimaxConfig table
- server/migrate.ts: ADD COLUMN IF NOT EXISTS group_id
- server/routes/reels.ts: fetch and pass groupId alongside apiKey
- server/services/ffmpeg.ts: minimax_group_id in request interface/body
- ffmpeg-service/main.py: GroupId in URL, threaded through all call sites
- client/src/pages/settings.tsx: Group ID input in Minimax settings card
- Python: capture tts_error_msg on exception, return in response
- Python: log tts_provider, minimax_api_key presence before call
- ffmpeg.ts: read tts_error from response, log and return it
- reels.ts: store TTS error in post.generationError for visibility
- Add minimax_config table (migration + schema + storage CRUD)
- Add GET/POST /api/minimax/config routes
- Pass tts_provider + minimax_api_key through ffmpeg service
- Add generate_tts_minimax() in Python using Minimax T2A v2 API
- Fall back to ffsubsync for subtitle sync (no WordBoundary events)
- Add Minimax config card in Settings page
- Add provider toggle (Edge TTS / Minimax) + French voices in new-reel
Some voices/texts do not produce WordBoundary events from edge_tts.
When word_boundaries is empty, generate_ass_from_word_boundaries returns
without writing the ASS file, causing FFmpeg to fail with ENOENT.
Fallback to the old ffsubsync path when no boundaries are captured.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- ffmpeg-service/main.py: replace ffsubsync path with exact word-boundary timing from edge_tts for TTS subtitles. Karaoke styling preserved.
- server/services/ttsSync.ts: fix word count to match TTS-cleaned text, remove artificial punctuationPause subtraction, strip punctuation tokens from count.
- server/routes/reels.ts: remove redundant ttsSyncService calls in preview/background; word_duration is ignored by Python, these only wasted TTS generations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
MichaelandClaude Opus 4.7
Michael
Jacques