extern/easydiffusion
1
0
Fork 1
mirror of https://github.com/easydiffusion/easydiffusion.git synced 2025-08-08 07:24:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

Created ControlNet (markdown)

cmdr2
2023-08-01 18:27:49 +05:30
parent f423240429
commit 9b2f197eea
1 changed files with 37 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
ControlNet.md Normal file

@ -0,0 +1,37 @@
ControlNets allow you to select a image to guide the AI, to make it follow your control image more closely. You can pick a filter to pre-process the image, and one of the known (or custom) controlnet models.
While this sounds similar to image-to-image, ControlNets allow the AI to extract meaningful information from the image and make completely different images in the same style. For e.g. following the same body posture as your initial image, or the same color style or image composition.
**Quick guide to using ControlNets in Easy Diffusion**
1. Enable beta and diffusers: https://github.com/easydiffusion/easydiffusion/wiki/The-beta-channel#test-diffusers-in-beta
2. In the `Image Settings` panel, set a `Control Image`. This can be any image that you want the AI to follow. For e.g. download the attached painting (of a girl) and set that as the control image.
3. Then set `Filter to apply` to `Canny`. This will automatically select `Canny` as the controlnet model as well.
4. Type `Emma Watson` in the prompt box (at the top), and use `1808629740` as the seed, and `euler_a` with 25 steps and `SD 1.4` model (or any other Stable Diffusion model).
5. Make an image.
It'll take a while during the first image, since it'll download about 1.3 GB of models for the canny controlnet.
**Pose from images:**
You can also set a photo of a person in a particular pose, and make the AI follow that pose. For e.g.
1. Download the attached photo of a man, and set that as the control image
2. Set `Filter to apply` to `Open Pose` (the first one). This will automatically select `OpenPose` as the controlnet model.
3. Type `Knight in black armor` in the prompt box (at the top), and use `1873330527` as the seed, and `euler_a` with 25 steps, and `SD 1.4` model (or any other SD model).
5. Make an image.
**Custom pose images:**
You can also use images generated in the OpenPose format, e.g. from the civitai poses category. For e.g.
1. Download the attached openpose photo (stick figure), and set that as the control image.
2. Set `Filter to apply` to `None`.
3. Set `Model` to `Open Pose`
4. Type `Knight in black armor` in the prompt box (at the top), and use `1873330527` as the seed, and `euler_a` with 25 steps, and `SD 1.4` model (or any other SD model).
5. Make an image.
# Resources:
**Painting of a girl:**
![image](https://github.com/easydiffusion/easydiffusion/assets/844287/517c43a6-2253-4f92-a75b-b7f18a1e8581)
**Man posture:**
![image](https://github.com/easydiffusion/easydiffusion/assets/844287/46783770-a596-4708-9821-1152c0c3c63a)
**OpenPose image:**
![image](https://github.com/easydiffusion/easydiffusion/assets/844287/08d1c52c-7455-49fc-9651-352d062de7b9)