What is Control Net?

ControlNet is a neural network structure to control diffusion models by adding extra conditions. It clones the weights of a large diffusion model into a 'trainable copy' and a 'locked copy'. The 'trainable copy' learns task-specific conditions while preserving the original model's generation capabilities. This allows users to control image generation based on various input conditions, such as edge maps, segmentation maps, pose skeletons, and depth maps, offering precise control over the output image.