Executable server

README.md

@@ -1,13 +1,16 @@
-A simple way to install and use [Stable Diffusion](https://replicate.com/stability-ai/stable-diffusion) on your local computer. Provides a browser UI for generating images from text prompts. Just enter your text prompt, and see the generated image.
+# Stable Diffusion UI
+### A simple way to install and use [Stable Diffusion](https://replicate.com/stability-ai/stable-diffusion) on your own computer
 
-🎉 **New!** `img2img` is now supported! You can provide an image to generate new images based on it (and an optional text prompt). You can also use the generated image as the new input image in 1-click, to refine it further.
+---
+
+🎉 **New!** `img2img` and `inpaint` (masking) are now supported! You can provide an image to generate new images based on it (and an optional text prompt). You can also use the generated image as the new input image in 1-click, to refine it further. (Thanks [Andreas](https://github.com/andreasjansson)!)
 
 # What does this do?
 Two things:
-1. Automatically downloads and installs Stable Diffusion on your local computer (no need to mess with conda or environments)
+1. Automatically downloads and installs Stable Diffusion on your own computer (no need to mess with conda or environments)
 2. Gives you a simple browser-based UI to talk to your local Stable Diffusion. Enter text prompts and view the generated image. No API keys required.
 
-All the processing will happen on your local computer, it does not transmit your prompts or process on any remote server.
+All the processing will happen on your computer locally, it does not transmit your prompts or process on any remote server.
 
 <img src="https://github.com/cmdr2/stable-diffusion-ui/raw/main/media/shot-v4.jpg" height="500" alt="Screenshot of tool">
 
@@ -20,23 +23,34 @@ All the processing will happen on your local computer, it does not transmit your
 
 # Installation
 1. Clone this repository: `git clone https://github.com/cmdr2/stable-diffusion-ui.git` or [download the zip file](https://github.com/cmdr2/stable-diffusion-ui/archive/refs/heads/main.zip) and unzip.
-2. Open your terminal, and in the project directory run: `docker-compose up &` (warning: this will take some time during the first run, since it'll download Stable Diffusion's [docker image](https://replicate.com/stability-ai/stable-diffusion), nearly 17 GiB)
+2. Open your terminal, and in the project directory run: `./server` (warning: this will take some time during the first run, since it'll download Stable Diffusion's [docker image](https://replicate.com/stability-ai/stable-diffusion), nearly 17 GiB)
 3. Open http://localhost:8000 in your browser. That's it!
 
 If you're getting errors, please check the [Troubleshooting](#troubleshooting) section below.
 
+To stop the server, please run `./server stop`
+
 # Usage
-1. Open http://localhost:8000 in your browser (after running `docker-compose up &` from step 2 previously).
-2. Enter a text prompt, like `a photograph of an astronaut riding a horse` in the textbox.
-3. Press `Make Image`. This will take some time, depending on your system's processing power.
-4. See the image generated using your prompt.
-5. **New!** img2img: You can also choose an `initial image`, to generate an image based on that. An optional text prompt can help you refine this image.
+Open http://localhost:8000 in your browser (after running `./server` from step 2 previously).
+
+## With a text description
+1. Enter a text prompt, like `a photograph of an astronaut riding a horse` in the textbox.
+2. Press `Make Image`. This will take some time, depending on your system's processing power.
+3. See the image generated using your prompt.
+
+## With an image
+1. Click `Browse..` next to `Initial Image`. Select your desired image.
+2. An optional text prompt can help you further describe the kind of image you want to generate.
+3. Press `Make Image`. See the image generated using your prompt.
+
+You can also set an `Image Mask` for telling Stable Diffusion to draw in only the black areas in your image mask. White areas in your mask will be ignored.
 
 **Pro tip:** You can also click `Use as Input` on a generated image, to use it as the input image for your next generation. This can be useful for sequentially refining the generated image with a single click.
 
-Please [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues) if this did not work for you (after trying the common [troubleshooting](#troubleshooting) steps)!
+**Another tip:** Images with the same aspect ratio of your generated image work best. E.g. 1:1 if you're generating images sized 512x512.
 
-To stop the server, please run `docker-compose down`
+## Problems?
+Please [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues) if this did not work for you (after trying the common [troubleshooting](#troubleshooting) steps)!
 
 # Advanced Settings
 You can also set the configuration like `seed`, `width`, `height`, `num_outputs`, `num_inference_steps` and `guidance_scale` using the 'show' button next to 'Advanced settings'.
@@ -58,7 +72,7 @@ If you have an NVIDIA GPU and the latest [NVIDIA driver](http://www.nvidia.com/D
 ## Some other process is already running at port 8000 / port 8000 could not be bound
 You can override the port used. Please change `docker-compose.yml` inside the project directory, and update the line `8000:8000` to `1337:8000` (or where 1337 is whichever port number you want).
 
-After doing this, please restart your server, by running `docker-compose down` and then `docker-compose up &`.
+After doing this, please restart your server, by running `./server restart`.
 
 After this, you can access the server at `http://localhost:1337` (where 1337 is the new port you specified earlier).
 

index.html

@@ -30,10 +30,10 @@
             width: 95%;
         }
     }
-    #init_image_preview_container {
+    .image_preview_container {
         display: none;
     }
-    #init_image_clear {
+    .image_clear_btn {
         position: absolute;
         transform: translateX(-50%);
         background: black;
@@ -87,9 +87,17 @@
     <textarea id="prompt">a photograph of an astronaut riding a horse</textarea><br/>
 
     <label for="init_image"><b>Initial Image:</b> (optional) </label> <input id="init_image" name="init_image" type="file" /> </button><br/>
-    <div id="init_image_preview_container">
+    <div id="init_image_preview_container" class="image_preview_container">
         <img id="init_image_preview" src="" width="100" height="100" />
-        <button id="init_image_clear">X</button>
+        <button id="init_image_clear" class="image_clear_btn">X</button>
+    </div><br/>
+
+    <div id="mask_setting">
+        <label for="mask"><b>Image Mask:</b> (optional) </label> <input id="mask" name="mask" type="file" /> </button><br/>
+        <div id="mask_preview_container" class="image_preview_container">
+            <img id="mask_preview" src="" width="100" height="100" />
+            <button id="mask_clear" class="image_clear_btn">X</button>
+        </div>
     </div>
 
     <div id="configHeader"><b>Advanced settings:</b> [<a id="configToggleBtn" href="#">show</a>]</div>
@@ -133,6 +141,8 @@ let widthField = document.querySelector('#width')
 let heightField = document.querySelector('#height')
 let initImageSelector = document.querySelector("#init_image")
 let initImagePreview = document.querySelector("#init_image_preview")
+let maskImageSelector = document.querySelector("#mask")
+let maskImagePreview = document.querySelector("#mask_preview")
 let promptStrengthField = document.querySelector('#prompt_strength')
 let promptStrengthValueLabel = document.querySelector('#prompt_strength_value')
 
@@ -143,6 +153,10 @@ let initImagePreviewContainer = document.querySelector('#init_image_preview_cont
 let initImageClearBtn = document.querySelector('#init_image_clear')
 let promptStrengthContainer = document.querySelector('#prompt_strength_container')
 
+let maskSetting = document.querySelector('#mask_setting')
+let maskImagePreviewContainer = document.querySelector('#mask_preview_container')
+let maskImageClearBtn = document.querySelector('#mask_clear')
+
 let showConfigToggle = document.querySelector('#configToggleBtn')
 let configBox = document.querySelector('#config')
 let outputMsg = document.querySelector('#outputMsg')
@@ -229,6 +243,10 @@ async function makeImage() {
     if (initImagePreview.src.indexOf('data:image/png;base64') !== -1) {
         reqBody['init_image'] = initImagePreview.src
         reqBody['prompt_strength'] = promptStrengthField.value / 10
+
+        if (maskImagePreview.src.indexOf('data:image/png;base64') !== -1) {
+            reqBody['mask'] = maskImagePreview.src
+        }
     }
 
     let res = ''
@@ -321,6 +339,8 @@ async function makeImage() {
             initImagePreviewContainer.style.display = 'block'
             promptStrengthContainer.style.display = 'block'
 
+            maskSetting.style.display = 'block'
+
             randomSeedField.checked = false
             seedField.value = seed
             seedField.disabled = false
@@ -380,6 +400,7 @@ function showInitImagePreview() {
     if (initImageSelector.files.length === 0) {
         initImagePreviewContainer.style.display = 'none'
         promptStrengthContainer.style.display = 'none'
+        maskSetting.style.display = 'none'
         return
     }
 
@@ -391,6 +412,8 @@ function showInitImagePreview() {
         initImagePreview.src = reader.result
         initImagePreviewContainer.style.display = 'block'
         promptStrengthContainer.style.display = 'block'
+
+        maskSetting.style.display = 'block'
     })
 
     if (file) {
@@ -402,11 +425,46 @@ showInitImagePreview()
 
 initImageClearBtn.addEventListener('click', function() {
     initImageSelector.value = null
+    maskImageSelector.value = null
+
     initImagePreview.src = ''
+    maskImagePreview.src = ''
+
     initImagePreviewContainer.style.display = 'none'
+    maskImagePreviewContainer.style.display = 'none'
+
+    maskSetting.style.display = 'none'
+
     promptStrengthContainer.style.display = 'none'
 })
 
+function showMaskImagePreview() {
+    if (maskImageSelector.files.length === 0) {
+        maskImagePreviewContainer.style.display = 'none'
+        return
+    }
+
+    let reader = new FileReader()
+    let file = maskImageSelector.files[0]
+
+    reader.addEventListener('load', function() {
+        maskImagePreview.src = reader.result
+        maskImagePreviewContainer.style.display = 'block'
+    })
+
+    if (file) {
+        reader.readAsDataURL(file)
+    }
+}
+maskImageSelector.addEventListener('change', showMaskImagePreview)
+showMaskImagePreview()
+
+maskImageClearBtn.addEventListener('click', function() {
+    maskImageSelector.value = null
+    maskImagePreview.src = ''
+    maskImagePreviewContainer.style.display = 'none'
+})
+
 setInterval(healthCheck, HEALTH_PING_INTERVAL * 1000)
 </script>
 

main.py

@@ -13,6 +13,7 @@ app = FastAPI()
 class ImageRequest(BaseModel):
     prompt: str
     init_image: str = None # base64
+    mask: str = None # base64
     num_outputs: str = "1"
     num_inference_steps: str = "50"
     guidance_scale: str = "7.5"
@@ -51,6 +52,9 @@ async def image(req : ImageRequest):
         data['input']['init_image'] = req.init_image
         data['input']['prompt_strength'] = req.prompt_strength
 
+        if req.mask is not None:
+            data['input']['mask'] = req.mask
+
     if req.seed == "-1":
         del data['input']['seed']
 

server

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+CMD="$1"
+if [ -z "$1" ]; then
+  CMD="start"
+fi
+
+start_server() {
+  docker-compose up &
+}
+
+stop_server() {
+  docker-compose down
+}
+
+if [ "$CMD" == "start" ]; then
+  start_server
+elif [ "$CMD" == "stop" ]; then
+  stop_server
+elif [ "$CMD" == "restart" ]; then
+  stop_server
+  start_server
+else
+  echo "Unknown option: $1 (Expected start or stop)"
+fi