Design systematic video codec constraint handling for resolution requirements #2
Labels
No labels
codec-constraints
enhancement
resolution
video-encoding
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kade/ComfyUI_Scriptable#2
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem: ComfyUI video encoding fails with libx264 when dimensions are not even (divisible by 2). Current fix hardcodes even dimension enforcement in VideoFromComponents.save_to(), which is not sustainable for future codecs.
Issues:
Codec Constraints:
Requirements:
Research Tasks:
Related Files:
Implementation Complete
What Was Implemented
Codec Constraint Protocol System (
comfy_api/latest/_util/codec_constraints.py)User Configuration (
VideoConfigdataclass)Integration in Video Encoding (
comfy_api/latest/_input_impl/video_types.py)Custom UI Widget (
web/scripts/resolution_selector.js)Configuration Nodes (
comfy_extras/nodes_video_config.py)Unit Tests (
tests-unit/comfy_api_test/codec_constraints_simple_test.py)Documentation (
docs/CODEC_CONSTRAINTS.md)Files Modified/Created
comfy_api/latest/_util/codec_constraints.pycomfy_api/latest/_input_impl/video_types.pycomfy_extras/nodes_video_config.pyweb/scripts/resolution_selector.jsweb/__init__.pytests-unit/comfy_api_test/codec_constraints_simple_test.pydocs/CODEC_CONSTRAINTS.mdcomfy_api/latest/_util/__init__.pyNext Steps
Debug Update
Issue Found:
From logs:
[VideoEncoding] Input resolution: 680x499, codec: autoTypeError: sin(): argument input (position 1) must be Tensor, not floatRoot Causes:
sin()instead oftorch.sin()- causes TypeErrorFix Applied:
Modified video_types.py to force H.264 constraint validation even when codec is "auto":
This ensures that even with codec="auto", the H.264 constraints (even dimensions) are enforced and cropping occurs.
Next Steps:
torch.sin()instead ofsin()HDMA Scanline Shift Investigation
Error Found:
From docker logs:
Root Cause:
The HDMA scanline shift pattern used
torch.roll(X, shift_amount.int(), dims=1)whereshift_amountis a tensor (same shape as Y). PyTorch'storch.roll()only supports shifting the entire tensor by a constant amount (tuple of ints), not per-row shifts.Fix Applied:
Replaced
torch.roll()with advanced indexing:This creates column indices, shifts them per-row using modulo arithmetic for wraparound, then uses advanced indexing to select the shifted values.
Files Fixed:
Committed:
The HDMA pattern should now work correctly.
Advanced Demoscene Patterns Research Complete
Created: ADVANCED_DEMOSCENE_PATTERNS.md
Research Sources:
8 Complex Patterns Implemented:
Domain Warped Fractal Flow - Multi-iteration domain distortion with fractal noise
Metaballs with Lissajous Motion - Organic blob rendering with smooth distance fields
Mandelbrot Set Zoom - Classic fractal with exponential zoom and color cycling
Flower of Life Sacred Geometry - Ancient overlapping circle pattern
Torus Knot Visualization - 3D distance field rendering of (3,2) trefoil knot
Julia Set Explorer - Dynamic Julia set with animated parameter path
Golden Ratio Spiral - Fibonacci spiral based on φ = (1+√5)/2
Voronoi Cell Shading - Animated Voronoi diagram with edge detection
All Patterns Include:
Committed:
Next Steps:
HDMA Scanline Shift Optimization Investigation
Problem Reported:
Root Cause Analysis:
Original implementation used advanced indexing:
Performance Issues:
Research Sources:
Optimization Strategies Considered:
Solution Implemented:
Replaced pixel shifting with coordinate distortion:
Benefits:
Files Created/Modified:
Committed:
Next Steps:
Domain Warped Fractal Flow Shape Error Fixed
Error from Docker Logs:
Root Cause:
The pattern used torch.stack([X, Y], dim=-1) which created 3D tensors with shape [512, 512, 2]. When operations were applied to these stacked tensors, they preserved the extra dimension, causing the final output to have shape [600, 512, 512, 2, 3] instead of the expected [600, 512, 512, 3].
Original Code:
Fixed Code:
Benefits:
Committed:
Next: Test the pattern again - should now produce correct animated output.
Domain Warped Fractal Flow Still Frame Investigation
Problem: User ran updated pattern but still got still frame with same shape error.
Root Cause Analysis:
Docker container not picking up git changes: The docker container is using the OLD version of ADVANCED_DEMOSCENE_PATTERNS.md that still has torch.stack code. The file inside the container shows:
Using ComfyUI_WolfSigmas: The user is running scripts through ComfyUI_WolfSigmas (wolf_scriptable_image.py), not ComfyUI_Scriptable. The error comes from:
Import error: ComfyUI_Scriptable has import order issue - SCRIPTABLE_NODES_ENABLED is defined after the nodes import, but nodes.py tries to import it from the parent package.
Fixed:
Action Required:
The user needs to manually copy the updated pattern from the docs to their script in ComfyUI, since the docker container is not picking up the git submodule changes.
Script to Test:
Copy the FIXED version from ADVANCED_DEMOSCENE_PATTERNS.md Pattern 1 (lines 53-70) which uses:
Committed:
Next: User should manually copy the fixed pattern to test.
Container Setup Fixed - User Running Old Code
Status: Container now loads ComfyUI_Scriptable successfully. Import order fixed.
Issue: User is still running the OLD pattern code with torch.stack, causing shape error [600, 512, 512, 2, 3].
Fixed Code Location: The corrected pattern is now in ADVANCED_DEMOSCENE_PATTERNS.md lines 53-70.
Action Required: Copy the NEW fixed code from the file:
Committed:
Next: User needs to copy the NEW code from the file to their ComfyUI script.
User Only Copied Partial Code - Missing Setup
Error from logs:
Root Cause: User only copied the for loop portion, not the setup code that defines total_frames, width, height, fps, duration, etc.
Action Required: Copy the COMPLETE pattern including all setup code from ADVANCED_DEMOSCENE_PATTERNS.md lines 40-80:
Next: Copy the complete code including all setup variables.