ControlNets allow you to select a image to guide the AI, to make it follow your control image more closely. You can pick a filter to pre-process the image, and one of the known (or custom) controlnet models.
While this sounds similar to image-to-image, ControlNets allow the AI to extract meaningful information from the image and make completely different images in the same style. For e.g. following the same body posture as your initial image, or the same color style or image composition.

Quick guide to using ControlNets in Easy Diffusion
- In the
Image Settings
panel, set a Control Image
. This can be any image that you want the AI to follow. For e.g. download this painting and set that as the control image.
- Then set
Filter to apply
to Canny
. This will automatically select Canny
as the controlnet model as well.
- Type
Emma Watson
in the prompt box (at the top), and use 1808629740
as the seed, and euler_a
with 25 steps and SD 1.4
model (or any other Stable Diffusion model).
- Make an image.
It'll take a while during the first image, since it'll download about 1.3 GB of models for the canny controlnet.
Pose from images:
You can also set a photo of a person in a particular pose, and make the AI follow that pose. For e.g.
- Download this photo of a man, and set that as the control image
- Set
Filter to apply
to Open Pose
(the first one). This will automatically select OpenPose
as the controlnet model.
- Type
Knight in black armor
in the prompt box (at the top), and use 1873330527
as the seed, and euler_a
with 25 steps, and SD 1.4
model (or any other SD model).
- Make an image.
Custom pose images:
You can also use images generated in the OpenPose format, e.g. from the civitai poses category. For e.g.
- Download this openpose photo (stick figure), and set that as the control image.
- Set
Filter to apply
to None
.
- Set
Model
to Open Pose
- Type
Knight in black armor
in the prompt box (at the top), and use 1873330527
as the seed, and euler_a
with 25 steps, and SD 1.4
model (or any other SD model).
- Make an image.
Resources: