This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

ScriptableNoise Node

Overview

The ScriptableNoise node allows you to write Python scripts that generate custom noise tensors for diffusion samplers. Unlike other scriptable nodes that generate immediate output, ScriptableNoise creates a noise generator object that will be called by the sampler during the diffusion process.

What is Noise in Diffusion?

In diffusion models, noise represents random perturbations added to images during the forward diffusion process. The reverse diffusion process (generating images) starts from pure noise and gradually denoises it to create a coherent image.

The noise tensor has the same shape as the latent: [batch, channels, height, width] where channels is typically 4 for stable diffusion models.

How ScriptableNoise Works Differently

Unlike other scriptable nodes that execute immediately and return a value, ScriptableNoise:

1. Executes your script once to create a noise generator object
2. Returns this generator object to ComfyUI
3. The sampler calls the generator's generate_noise() method during sampling
4. Each call can produce different noise based on the current latent

This allows the noise to adapt based on the latent being processed, enabling dynamic noise generation.

Inputs

Parameter	Type	Default	Range	Description
seed	INT	0	0-18446744073709551615	Random seed for reproducibility
device_selection	COMBO	"AUTO"	AUTO, CPU, GPU	Device to generate noise on
model	MODEL	-	-	Optional model for device detection
script	STRING	(see below)	-	Python script to execute

Outputs

• noise: A noise generator object with a generate_noise(latent_image_dict) method

Default Script

# Script to generate custom noise
# Available: latent_image (dict with 'samples'), seed, model, device, torch, math
# MUST assign 'output_noise_tensor' (same shape as input_samples)

noise = torch.zeros_like(input_samples, device=torch.device(device))
output_noise_tensor = noise

How the Default Script Works

noise = torch.zeros_like(input_samples, device=torch.device(device))
output_noise_tensor = noise

• torch.zeros_like(): Creates a tensor of zeros with the same shape as input_samples
• input_samples: The latent tensor from the sampler
• device=torch.device(device): Places the tensor on the specified device (CPU or GPU)
• output_noise_tensor: The required output variable name

This creates zero noise, which means the sampler will use no random perturbations.

Available Variables in generate_noise()

When the sampler calls generate_noise(latent_image_dict), your script has access to:

• latent_image: The full latent dictionary from ComfyUI
• input_samples: The latent tensor (same as latent_image['samples'])
• seed: The seed value from the node
• model: The model object (if connected)
• device: The device string (e.g., "cuda:0", "cpu")
• torch: The PyTorch library
• math: The Python math library

Example: Standard Gaussian Noise

# Generate standard Gaussian noise (same as ComfyUI's default)
torch.manual_seed(seed)
noise = torch.randn_like(input_samples, device=torch.device(device))
output_noise_tensor = noise

Explanation:

• torch.manual_seed(seed): Sets random seed for reproducibility
• torch.randn_like(): Generates random numbers from normal distribution (mean=0, std=1)
• Same shape as input_samples, same device
• This is equivalent to ComfyUI's default noise generation

Example: Colored Noise (Frequency-Filtered)

import torch
import torch.nn.functional as F

torch.manual_seed(seed)

# Generate base noise
noise = torch.randn_like(input_samples, device=torch.device(device))

# Apply frequency filter in Fourier domain
# Convert to frequency domain
noise_fft = torch.fft.fftn(noise)

# Create a low-pass filter (keep low frequencies, attenuate high frequencies)
height, width = noise.shape[-2:]
y_coords = torch.linspace(-1, 1, height, device=torch.device(device))
x_coords = torch.linspace(-1, 1, width, device=torch.device(device))
Y, X = torch.meshgrid(y_coords, x_coords, indexing='ij')
filter_mask = torch.exp(-(X**2 + Y**2) / (2 * 0.5**2))  # Gaussian low-pass
filter_mask = filter_mask.unsqueeze(0).unsqueeze(0).expand_as(noise_fft.real)

# Apply filter
filtered_fft = noise_fft * filter_mask

# Convert back to spatial domain
colored_noise = torch.fft.ifftn(filtered_fft).real

output_noise_tensor = colored_noise

Explanation:

• Generates standard noise first
• Converts to frequency domain using FFT
• Creates a Gaussian low-pass filter in frequency space
• Multiplies the noise by the filter to attenuate high frequencies
• Converts back to spatial domain using inverse FFT
• Result: "colored" noise with more low-frequency content (smoother, larger structures)

Mathematical Background: Fourier Transform

The Fourier transform converts a signal from the spatial domain (pixels) to the frequency domain (frequencies):

• Low frequencies: Slow variations (large structures, smooth gradients)
• High frequencies: Fast variations (fine details, sharp edges)

Filtering in the frequency domain allows you to control the characteristics of noise:

• Low-pass filter: Keeps low frequencies, removes high frequencies → smoother noise
• High-pass filter: Keeps high frequencies, removes low frequencies → sharper, more detailed noise
• Band-pass filter: Keeps a specific frequency range → noise with specific scale

The Gaussian filter in frequency space:

H(f) = exp(-f² / (2σ²))

Where f is the frequency and σ controls the cutoff frequency.

Example: Perlin-like Noise

import torch

torch.manual_seed(seed)

# Generate multi-scale noise (octaves)
octaves = 4
colored_noise = torch.zeros_like(input_samples, device=torch.device(device))

for i in range(octaves):
    # Generate noise at different scales
    scale = 2 ** i
    # Downsample (simple approximation)
    h, w = input_samples.shape[-2:]
    small_h, small_w = max(1, h // scale), max(1, w // scale)
    
    # Generate noise at this scale
    small_noise = torch.randn(input_samples.shape[0], input_samples.shape[1], 
                             small_h, small_w, device=torch.device(device))
    
    # Upsample back to original size
    upsampled = F.interpolate(small_noise.unsqueeze(1), size=(h, w), 
                               mode='bilinear', align_corners=False).squeeze(1)
    
    # Add to total with decreasing amplitude
    amplitude = 1.0 / (2 ** i)
    colored_noise += upsampled * amplitude

# Normalize to match original noise variance
colored_noise = colored_noise / torch.sqrt(torch.tensor(octaves))

output_noise_tensor = colored_noise

Explanation:

• Generates noise at multiple scales (octaves)
• Each octave has half the resolution of the previous
• Lower octaves (larger scales) contribute more
• Higher octaves (smaller scales) contribute less
• Result: noise with structure at multiple scales (similar to Perlin noise)

Mathematical Background: Octave Synthesis

Perlin noise and similar algorithms synthesize noise by summing multiple octaves:

N(x) = Σ(2⁻ⁱ · n(2ⁱ · x))

Where:

• i is the octave index
• 2⁻ⁱ is the amplitude (decreases with each octave)
• 2ⁱ · x is the scaled coordinate
• n() is a base noise function

This creates noise with fractal-like properties: similar patterns at different scales.

Example: Anisotropic Noise (Directional)

import torch

torch.manual_seed(seed)

# Generate base noise
noise = torch.randn_like(input_samples, device=torch.device(device))

# Apply directional smoothing (anisotropic)
# Smooth more in one direction than another
kernel_size = 5
sigma_x = 0.5  # Less smoothing in x
sigma_y = 2.0  # More smoothing in y

# Create anisotropic Gaussian kernel
y = torch.arange(-kernel_size//2, kernel_size//2 + 1, device=torch.device(device), dtype=torch.float32)
x = torch.arange(-kernel_size//2, kernel_size//2 + 1, device=torch.device(device), dtype=torch.float32)
Y, X = torch.meshgrid(y, x, indexing='ij')
kernel = torch.exp(-(X**2 / (2 * sigma_x**2) + Y**2 / (2 * sigma_y**2)))
kernel = kernel / kernel.sum()  # Normalize

# Reshape kernel for convolution
kernel = kernel.view(1, 1, kernel_size, kernel_size)
kernel = kernel.expand(input_samples.shape[1], 1, kernel_size, kernel_size)

# Apply convolution to each channel
anisotropic_noise = torch.zeros_like(noise)
for c in range(input_samples.shape[1]):
    for b in range(input_samples.shape[0]):
        anisotropic_noise[b, c] = F.conv2d(
            noise[b, c:c+1], 
            kernel[c:c+1], 
            padding=kernel_size//2
        )

output_noise_tensor = anisotropic_noise

Explanation:

• Creates a Gaussian kernel with different sigmas in x and y directions
• sigma_x < sigma_y: Less smoothing horizontally, more vertically
• Convolves the noise with this anisotropic kernel
• Result: noise that's stretched in one direction

Mathematical Background: Anisotropic Gaussian

The anisotropic Gaussian function:

G(x, y) = exp(-(x² / (2σₓ²) + y² / (2σᵧ²)))

When σₓ ≠ σᵧ, the Gaussian is stretched in one direction:

• σₓ < σᵧ: Stretched horizontally (wider in x)
• σₓ > σᵧ: Stretched vertically (wider in y)

This creates noise with directional characteristics, which can be useful for creating directional textures or patterns.

Example: Latent-Dependent Noise

import torch

# Generate noise that depends on the latent content
# This can create more coherent or structured results

# Calculate statistics of the input latent
mean = input_samples.mean(dim=[2, 3], keepdim=True)
std = input_samples.std(dim=[2, 3], keepdim=True)

# Generate noise scaled by latent statistics
torch.manual_seed(seed)
base_noise = torch.randn_like(input_samples, device=torch.device(device))

# Scale noise by latent standard deviation (noisier areas get more noise)
scaled_noise = base_noise * (std / std.mean())

output_noise_tensor = scaled_noise

Explanation:

• Calculates mean and standard deviation of the input latent
• Scales the noise by the local standard deviation
• Areas with higher variation get more noise
• Result: noise that adapts to the latent structure

Example: Time-Varying Noise (for Animation)

import torch

# For animation, you might want noise that varies over time
# This requires a time parameter, which you can pass via the latent

# Extract a "time" parameter from the latent (e.g., from batch index)
# This is a hack - in practice, you'd need a proper time parameter
batch_index = 0  # In practice, you'd get this from somewhere

torch.manual_seed(seed + batch_index)

# Generate noise with time-varying characteristics
time_factor = batch_index / 100.0  # Normalize time

# Mix different noise types based on time
noise1 = torch.randn_like(input_samples, device=torch.device(device))
noise2 = torch.randn_like(input_samples, device=torch.device(device))

# Blend based on time
blended_noise = (1 - time_factor) * noise1 + time_factor * noise2

output_noise_tensor = blended_noise

Explanation:

• This example shows how noise could vary over time for animation
• Uses the batch index as a proxy for time
• Blends between two different noise patterns based on time
• In practice, you'd need a proper time parameter from the workflow

Device Selection

The device_selection parameter controls where noise is generated:

• AUTO: Uses the model's device if available, otherwise GPU if available, otherwise CPU
• CPU: Forces CPU generation (slower but compatible)
• GPU: Forces GPU generation (faster but requires CUDA)

Error Handling

The node includes error handling:

• Syntax errors in the script are caught during compilation
• Runtime errors during noise generation return zero noise
• Invalid output types or shapes are caught
• Errors fall back to zero noise

Security Considerations

This node uses exec() to run your script. This is intentionally powerful for local development but should not be used in production or exposed to untrusted users.

Related Nodes

• ScriptableLatent: For generating latents directly
• KSampler / KSamplerAdvanced: Samplers that use noise
• Random Noise node: ComfyUI's standard noise generator