AI Reels Creator

AI Reels Creator is a local, GPU-accelerated production studio for turning an original character, a story, and a voice into polished vertical social videos.

It is built for creators who want more than unrelated AI images. Each episode is treated as a production: plan the story, preserve character identity, carry visual continuity from shot to shot, generate approved keyframes and animation clips, create dialogue, mix audio, add captions, and render a finished Reel.

The project is local-first. Scripts, references, drafts, generated media, voice settings, and model inference stay on the server. This public repository contains code, generic configuration, tests, workflow structure, and documentation. It intentionally excludes credentials, private IP addresses, hostnames, usernames, uploaded references, generated episodes, model binaries, and machine logs.

• Creates episodes from a rough concept, prose story, dialogue script, or numbered shot list.
• Supports explicit-scene planning that preserves the user’s order and supplied fields.
• Supports automatic story planning when the user provides only a general concept.
• Produces an editable storyboard with timing, action, camera, dialogue, visual prompts, audio cues, and approval state.
• Maintains an Official Dez Reference Set with one primary reference and optional secondary references.
• Locks Dez’s character profile into every image request so scenes preserve species, proportions, clothing, colors, accessories, and style.
• Builds a canonical episode plan with shared locations, persistent props, lighting, previous-scene state, and allowed continuity changes.
• Generates vertical cartoon keyframes through ComfyUI and sends approved keyframes to image-to-video workflows.
• Generates dialogue and narration with a provider interface and persistent Chatterbox Dez voice profile.
• Provides scene-level audio preview so speech can be heard with the generated clip.
• Supports optional local lip sync without making final rendering depend on it.
• Creates SRT and optional burned-in captions from known dialogue timing.
• Mixes dialogue, narration, sound effects, and user-owned or royalty-free music.
• Assembles a compatible 720x1280, 30 FPS H.264/AAC Instagram Reel.
• Keeps versions instead of silently overwriting generated assets.
• Persists drafts, job IDs, selected versions, references, voice settings, and episode state across refreshes and restarts.
• Provides diagnostics for services, GPU memory, disk space, workflow validation, active jobs, and recent errors.

text Browser on the private LAN | v Gradio web application :7860 | v Python orchestration services | | | v v v ComfyUI API Chatterbox FFmpeg/ffprobe :8188 isolated TTS media pipeline | v GPU image and image-to-video workflows | v Episode JSON state and versioned media

The web application and ComfyUI remain separate services. Dez Studio communicates with ComfyUI over HTTP and WebSocket progress APIs, with history polling as a fallback. Long-running work is queued so the Gradio interface remains usable.

The application has seven focused tabs:

1. New Episode: title, story/script, reference uploads, duration, style, voice, music, captions, and generation toggles.
2. Storyboard: editable scene fields, approval, timing, dialogue, camera, motion, sound effects, and transitions.
3. Scene Images: grouped versions, prompts, seeds, references, approvals, rejection, deletion, and regeneration.
4. Animation: approved keyframes, motion prompts, generated clips, playback, approval, and rejection.
5. Audio: exact text-to-speech input, voice preview, scene audio plus video preview, music, effects, and ducking.
6. Final Reel: clip order, transitions, captions, audio levels, 720x1280 render, validation, and playback.
7. System: service status, URLs, GPU, VRAM, disk, jobs, errors, models, and workflow checks.

Dez is represented by a persistent character profile, not a newly invented description for every scene. Every image request carries:

• character ID and locked state
• primary reference asset ID
• enabled reference asset IDs
• reference-set version or hash
• global cartoon style
• canonical appearance description
• negative consistency constraints

The default profile describes Dez as an original anthropomorphic cartoon squirrel with warm medium-brown fur, tan muzzle and tail underside, expressive cartoon eyes, buck teeth, rounded muzzle, black paisley bandana, black DEZ NUTS hoodie, black shorts, silver chain, black-and-white sneakers, and a large curled bushy tail. The editable source is characters/dez/character_bible.md.

References are stored by server asset ID, filename, hash, thumbnail, enabled state, and order. Raw browser File objects are never placed in localStorage. A reference version is included in jobs so an old result cannot replace a newer reference set. Official production references are not committed to Git.

Before image generation, the approved storyboard is normalized into a canonical episode plan containing:

• episode summary and aspect ratio
• rendering style
• shared location and time of day
• lighting direction
• persistent props and screen content
• character start position
• per-scene purpose, action, camera, expression, start state, and end state
• previous-scene relationship
• explicitly allowed continuity changes

Scenes are generated in numeric story order. Scene 002 inherits scene 001’s end state, scene 003 inherits scene 002’s end state, and so on. The room, desk, monitor, chair, lighting, clothing, props, and style remain unchanged unless the storyboard explicitly changes them.

The shared image prompt is assembled as:

text global cartoon style locked Dez identity and references shared episode setting and persistent props previous-scene end state current action and expression camera and lighting negative constraints

Dialogue goes to audio and caption processing, not image prompts. Negative constraints reject white studio backgrounds, reference sheets, speech bubbles, rendered dialogue, poster layouts, photorealism, live-action humans, duplicated characters, and unrelated compositions.

Drafts use the versioned browser key:

text dez-studio:draft:v1

Persisted JSON includes draft ID, schema version, update time, title and prompt, scenes and edits, style and audio settings, toggles, uploaded asset IDs and filenames, character profile, voice profile, generation IDs, selected versions, and episode state.

Autosave uses a 400 ms debounce. The UI shows Saved or Unsaved status and supports Save Draft and Clear/New Draft with confirmation. Stored data is validated and migrated before use. Corrupt or incompatible local storage is ignored safely.

Every generation request includes a snapshot of the active character/reference and voice versions. If references, voice settings, or scene edits change while a job is running, the response is rejected or marked stale rather than overwriting the current draft.

text /opt/dez-studio/ ├── app.py ├── config.yaml ├── requirements.txt ├── requirements-chatterbox.txt ├── .env.example ├── characters/dez/character_bible.md ├── workflows/image/ ├── workflows/image_to_video/ ├── services/ ├── scripts/ ├── tests/ ├── episodes/ ├── assets/ ├── logs/ └── temp/

Each episode is isolated:

text episodes/<episode-slug>/ ├── episode.json ├── script.txt ├── storyboard.json ├── references/ ├── keyframes/scene-001/ ├── clips/{raw,interpolated,upscaled,lip_synced}/ ├── audio/{dialogue,narration,sound_effects,music,mixed}/ ├── captions/ ├── previews/ ├── final/ ├── temp/ └── logs/

Generated assets use versioned names such as scene-001-image-v001.png, scene-001-clip-v001.mp4, and scene-001-dialogue-v001.wav. Selected versions are recorded in episode.json.

• Ubuntu Linux with systemd
• NVIDIA GPU with a working driver and CUDA-capable PyTorch in ComfyUI
• 24 GB VRAM is a practical target for conservative image-to-video defaults
• Enough disk for models, caches, episodes, and backups
• LAN access from the browser machine

The default pipeline uses 576x1024 working images, batch size one, 3–6 second clips, and a 720x1280 final Reel. Higher resolutions and longer clips require more VRAM, time, and disk.

text git git-lfs curl wget ffmpeg jq rsync unzip build-essential python3-dev python3-venv pkg-config libgl1 libglib2.0-0 fonts-dejavu-core

Verify:

bash ffmpeg -version ffprobe -version git lfs version

The main environment is separate from ComfyUI:

text gradio>=5.49,<7 PyYAML>=6.0 requests>=2.31 websocket-client>=1.8 python-dotenv>=1.0 psutil>=6.0 Pillow>=10.0 pytest>=8.0

Chatterbox is installed separately because its audio and PyTorch dependencies can conflict with ComfyUI:

text chatterbox-tts==0.1.7

Optional lip-sync runners also use separate environments and model requirements.

The repository assumes ComfyUI is already installed as a separate service. Do not replace a working ComfyUI environment during routine installation.

`bash sudo apt-get update sudo apt-get install -y git git-lfs curl wget ffmpeg jq rsync unzip build-essential
python3-dev python3-venv pkg-config libgl1 libglib2.0-0 fonts-dejavu-core

cd /opt/dez-studio python3 -m venv venv ./venv/bin/python -m pip install --upgrade pip ./venv/bin/pip install -r requirements.txt cp .env.example .env `

Edit config.yaml for non-secret settings and the ignored .env for provider URLs, optional authentication, and local runner paths. Never commit .env.

For Chatterbox:

bash python3 -m venv /opt/dez-studio/chatterbox-venv /opt/dez-studio/chatterbox-venv/bin/pip install -r requirements-chatterbox.txt

Development startup:

bash /opt/dez-studio/scripts/start-dev.sh

Service startup:

bash sudo systemctl daemon-reload sudo systemctl enable dez-studio sudo systemctl restart dez-studio sudo systemctl status dez-studio --no-pager

Safe defaults in config.yaml:

Environment variables include:

text DEZ_STUDIO_CONFIG COMFYUI_BASE_URL STORYBOARD_PROVIDER STORYBOARD_API_BASE STORYBOARD_API_KEY STORYBOARD_MODEL OLLAMA_BASE_URL OLLAMA_MODEL OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL TTS_PROVIDER DEZ_STUDIO_USERNAME DEZ_STUDIO_PASSWORD DEZ_STUDIO_DRAFT_SECRET LIPSYNC_COMMAND LATENTSYNC_CHECKPOINT LATENTSYNC_CONFIG LATENTSYNC_VAE_PATH

Secrets are read from the environment only and are not printed in normal logs.

The API client loads workflow JSON, validates node IDs and class types, injects values by explicit node ID, submits through the prompt API, follows WebSocket progress when available, and falls back to history polling.

Checked-in workflows:

• workflows/image/dez-keyframe-api.json
• workflows/image_to_video/dez-i2v-api.json

The design targets a Z-Image Turbo keyframe stack and a Wan 2.2 image-to-video stack with conservative 4090-compatible settings. Actual quality depends on installed models, filenames, VRAM, and custom nodes. Model binaries are intentionally not stored in Git.

Run:

bash /opt/dez-studio/scripts/validate-install.sh

If a model or node is missing, validation reports it instead of silently generating an unrelated image. IPAdapter, depth, interpolation, upscaling, and lip-sync integrations are optional.

Dez dialogue uses a persistent original-character voice profile. The default Chatterbox profile targets broad traits—bright, youthful, quick, mischievous, expressive, crisp, and intelligible—without imitating a named actor or copyrighted character.

The profile stores provider, model, reference audio ID, stable speaker seed, synthesis settings, emotion, post-processing, and voice version. Supported emotion overrides are amused, skeptical, concerned, annoyed, deadpan, excited, and quietly suspicious. Emotion changes energy and pacing, not speaker identity.

Post-processing is conservative: modest upward pitch shift, mild formant adjustment, controlled speed, light compression, de-essing, high-pass filtering, loudness normalization, and clipping protection. The same profile is used for preview, scene audio, lip-sync input, regeneration, and final render.

Optional reference audio should be an original 10–20 second recording with one speaker, no music, no reverb, and minimal background noise. Never use audio from an existing character or actor.

Lip sync is optional and isolated. When a compatible runner is available, the UI keeps mouth lip sync enabled; if it fails, final assembly uses the exact dialogue audio and preserves the original clip. Cartoon side profiles, occlusions, extreme expressions, and low-resolution faces can reduce quality.

Captions are generated from storyboard dialogue timing. The application writes SRT and can burn safe-margin ASS captions using Clean, Meme, Bold, or Subtitle presets.

`text approved clips

• dialogue and narration
• sound effects
• background music with optional ducking
• captions -> normalized 720x1280 H.264/AAC MP4 `

FFmpeg normalizes dimensions, frame rate, pixel format, audio sample rate, and stream layout. ffprobe validates duration, resolution, FPS, codecs, and file size before completion is reported. Add only user-owned or appropriately licensed music and effects.

Dez Studio is intended for a private LAN or protected Tailscale network, not the public internet.

• Bind Dez Studio to 0.0.0.0:7860 for LAN browser access.
• Bind ComfyUI to 0.0.0.0:8188 only when LAN access is needed.
• Restrict both ports to the detected LAN subnet or Tailscale interface.
• Do not create unrestricted firewall rules.
• Enable Gradio authentication through environment variables when appropriate.
• Keep API keys and passwords in an ignored environment file.
• Never commit references, voice recordings, episodes, logs, or model files.

After verifying the actual subnet:

bash sudo ufw allow from LAN_SUBNET_CIDR to any port 7860 proto tcp sudo ufw allow from LAN_SUBNET_CIDR to any port 8188 proto tcp

Check listeners with:

bash ss -ltnp | grep -E ':7860|:8188'

The repository deliberately does not publish a machine-specific address. Use the server’s LAN address with port 7860 for Dez Studio and port 8188 for ComfyUI.

1. Open Dez Studio on the private LAN.
2. Create an episode.
3. Upload the official Dez primary and secondary references.
4. Confirm Character locked.
5. Paste the story, concept, or exact dialogue script.
6. Build, edit, and approve the storyboard.
7. Generate scene images in story order.
8. Review, approve, reject, or delete image versions.
9. Animate approved keyframes and play the clips.
10. Type or edit exact dialogue in Audio and generate the selected voice.
11. Build scene audio plus video and listen to the combined clip.
12. Review captions, music, effects, and transitions.
13. Render the 720x1280 final Reel.
14. Play the final Reel and validate it with ffprobe.

`bash sudo systemctl status comfyui dez-studio --no-pager sudo systemctl restart comfyui dez-studio journalctl -u dez-studio -n 200 --no-pager journalctl -u comfyui -n 200 --no-pager

/opt/dez-studio/scripts/status.sh /opt/dez-studio/scripts/validate-install.sh /opt/dez-studio/scripts/create-test-episode.sh /opt/dez-studio/scripts/backup-config.sh

cd /opt/dez-studio ./venv/bin/pytest tests -q

curl http://127.0.0.1:8188/system_stats curl -I http://127.0.0.1:7860 ffprobe -v error -show_format -show_streams path/to/final.mp4 `

If refresh restores an old project, use the draft loader, choose the intended saved draft, and verify Saved status. Clear/New Draft writes a clear marker. A hard refresh is useful after a service update.

If Save Draft or refresh shows errors, inspect the Dez Studio journal and browser console. The draft schema is validated and migrated; malformed local storage is discarded safely. Reload after restarting so the Gradio event map matches the running application.

If Generate Selected Scene appears idle, verify storyboard approval, valid scene order, an official primary reference, and ComfyUI health. Long jobs run in the background and record job ID, prompt ID, progress, and failure reason in the episode log.

If images contain Dez but do not continue the story, regenerate after approving the canonical episode plan. It requires previous-scene inheritance, persistent props, and explicit location/time changes.

If the voice sounds stale or sliders appear ineffective, confirm the locked Chatterbox profile is selected rather than a cached old file. Change voice version or reset to the official profile to invalidate cache entries.

If the clip has no audible dialogue, play the generated dialogue first, then use Build Scene Audio + Video. Verify the mixed output with ffprobe. The source clip remains available if lip sync fails.

If the final Reel is blank, stale, or the wrong size, refresh Final Reel, confirm approved clips, assemble a preview, and render again. The target is 720x1280 at 30 FPS; changing an approval does not silently rewrite an existing render.

If a ComfyUI workflow fails, run validate-install.sh, confirm model filenames and custom nodes, inspect ComfyUI logs, and verify explicit node mappings.

Use:

bash /opt/dez-studio/scripts/backup-config.sh

Backups include application source, configuration, workflow JSON, character documentation, systemd files, and episode metadata while excluding large generated media and model binaries by default.

For updates: back up first, review the diff, keep local data outside commits, run tests, run validation, verify systemd, restart Dez Studio, and perform one low-cost test job. Do not replace working CUDA/PyTorch environments, delete models, or remove custom nodes during routine updates.

The suite covers storyboard modes, continuity and state inheritance, character and reference versioning, draft restore and clearing, workflow validation, mocked ComfyUI submission, image payloads, voice persistence, stale jobs, ffprobe parsing, captions, FFmpeg commands, episode state, scene-version grouping, rejection, and duplicate prevention.

bash ./venv/bin/pytest tests -q

The complete image-to-video result depends on models and custom nodes installed on the target ComfyUI server. The software pipeline validates requests and reports missing dependencies instead of claiming success.

The public repository is intentionally code-only. It excludes:

• environment files and credentials
• local IP addresses, hostnames, and usernames
• official Dez references and voice recordings
• episode scripts and generated media
• logs, caches, previews, and final videos
• ComfyUI checkpoints, LoRAs, VAEs, and other model binaries

Before publishing additional files:

bash git status --short git ls-files rg -n -i --hidden --glob '!.git/**' '(api[_-]?key|password|token|authorization|BEGIN (RSA|OPENSSH|EC|PRIVATE)|192\\.168\\.|/home/|/Users/)' .

AI Reels Creator is a modular local production application. Image, animation, voice, and lip-sync quality remain model-dependent, while the episode, character, audio, persistence, and export workflow stays consistent.

Keep changes scoped, add regression tests for behavior changes, and never commit secrets or local media. Third-party models, custom nodes, TTS packages, music, sound effects, and lip-sync checkpoints have their own licenses. Add an explicit project license before redistributing the application commercially.