Fix - reduce the amount of VRAM occupied when the program starts up, this caused a regression and failures on GPUs with 4 gb or less of VRAM

UI setting for preventing browser autostart

CONTRIBUTING.md

@@ -13,12 +13,11 @@ If you would like to contribute to this project, there is a discord for dicussio
 This is in-flux, but one way to get a development environment running for editing the UI of this project is:
 (swap `.sh` or `.bat` in instructions depending on your environment, and be sure to adjust any paths to match where you're working)
 
-1) `git clone` the repository, e.g. to `/projects/stable-diffusion-ui-repo`
-2) Download the pre-built end user archive from the link on github, and extract it, e.g. to `/projects/stable-diffusion-ui-archive`
-3) `cd /projects/stable-diffusion-ui-archive` and run the script to set up and start the project, e.g. `start.sh`
-4) Check you can view and generate images on `localhost:9000`
-5) Close the server, and edit `/projects/stable-diffusion-ui-archive/scripts/on_env_start.sh`
-6) Comment out the lines near the bottom that copies the `files/ui` folder, e.g:
+1) Install the project to a new location using the [usual installation process](https://github.com/cmdr2/stable-diffusion-ui#installation), e.g. to `/projects/stable-diffusion-ui-archive`
+2) Start the newly installed project, and check that you can view and generate images on `localhost:9000`
+3) Next, please clone the project repository using `git clone` (e.g. to `/projects/stable-diffusion-ui-repo`)
+4) Close the server (started in step 2), and edit `/projects/stable-diffusion-ui-archive/scripts/on_env_start.sh` (or `on_env_start.bat`)
+5) Comment out the lines near the bottom that copies the `files/ui` folder, e.g:
   
   for `.sh`
 ```
@@ -33,13 +32,13 @@ REM @xcopy sd-ui-files\ui ui /s /i /Y
 REM @copy sd-ui-files\scripts\on_sd_start.bat scripts\ /Y
 REM @copy "sd-ui-files\scripts\Start Stable Diffusion UI.cmd" . /Y
 ``` 
-7) Comment out the line at the top of `/projects/stable-diffusion-ui-archive/scripts/on_sd_start.sh` that copies `on_env_start`. For e.g. `@copy sd-ui-files\scripts\on_env_start.bat scripts\ /Y`
+6) Next, comment out the line at the top of `/projects/stable-diffusion-ui-archive/scripts/on_sd_start.sh` (or `on_sd_start.bat`) that copies `on_env_start`. For e.g. `@rem @copy sd-ui-files\scripts\on_env_start.bat scripts\ /Y`
 8) Delete the current `ui` folder at `/projects/stable-diffusion-ui-archive/ui`
 9) Now make a symlink between the repository clone (where you will be making changes) and this archive (where you will be running stable diffusion):
 `ln -s /projects/stable-diffusion-ui-repo/ui /projects/stable-diffusion-ui-archive/ui`
 or for Windows
-`mklink /D \projects\stable-diffusion-ui-archive\ui \projects\stable-diffusion-ui-repo\ui` (link name first, source repo dir second)
-9) Run the archive again `start.sh` and ensure you can still use the UI.
+`mklink /J \projects\stable-diffusion-ui-archive\ui \projects\stable-diffusion-ui-repo\ui` (link name first, source repo dir second)
+9) Run the project again (like in step 2) and ensure you can still use the UI.
 10) Congrats, now any changes you make in your repo `ui` folder are linked to this running archive of the app and can be previewed in the browser.
 
 Check the `ui/frontend/build/README.md` for instructions on running and building the React code.
@@ -47,9 +46,5 @@ Check the `ui/frontend/build/README.md` for instructions on running and building
 ## Development environment for Installer changes
 Build the Windows installer using Windows, and the Linux installer using Linux. Don't mix the two, and don't use WSL. An Ubuntu VM is fine for building the Linux installer on a Windows host.
 
-1. Install Miniconda 3 or Anaconda.
-2. Install `conda install -c conda-forge -y conda-pack`
-3. Open the Anaconda Prompt. Do not use WSL if you're building for Windows.
-4. Run `build.bat` or `./build.sh` depending on whether you're in Windows or Linux.
-5. Compress the `stable-diffusion-ui` folder created inside the `dist` folder. Make a `zip` for Windows, and `tar.xz` for Linux (smaller files, and Linux users already have tar).
-6. Make a new GitHub release and upload the Windows and Linux installer builds.
+1. Run `build.bat` or `./build.sh` depending on whether you're in Windows or Linux.
+2. Make a new GitHub release and upload the Windows and Linux installer builds created inside the `dist` folder.

README.md

@@ -1,49 +1,77 @@
-# Stable Diffusion UI v2
-### A simple 1-click way to install and use [Stable Diffusion](https://github.com/CompVis/stable-diffusion) on your own computer. No dependencies or technical knowledge required.
+# Stable Diffusion UI
+### Easiest way to install and use [Stable Diffusion](https://github.com/CompVis/stable-diffusion) on your own computer. No dependencies or technical knowledge required. 1-click install, powerful features.
+
+[![Discord Server](https://img.shields.io/discord/1014774730907209781?label=Discord)](https://discord.com/invite/u9yhsFmEkB) (for support, and development discussion) | [Troubleshooting guide for common problems](Troubleshooting.md)
+
+----
+
+## Step 1: Download the installer
 
 <p float="left">
   <a href="#installation"><img src="https://github.com/cmdr2/stable-diffusion-ui/raw/develop/media/download-win.png" width="200" /></a>
   <a href="#installation"><img src="https://github.com/cmdr2/stable-diffusion-ui/raw/develop/media/download-linux.png" width="200" /></a>
 </p>
 
-[![Discord Server](https://img.shields.io/discord/1014774730907209781?label=Discord)](https://discord.com/invite/u9yhsFmEkB) (for support, and development discussion) | [Troubleshooting guide for common problems](Troubleshooting.md)
+## Step 2: Run the program
+- On Windows: Double-click `Start Stable Diffusion UI.cmd`
+- On Linux: Run `./start.sh` in a terminal
 
-️‍🔥🎉 **New!** Use Custom Weights, Task Queue, Negative Prompt, Live Preview, More Samplers, In-Painting, Face Correction (GFPGAN) and Upscaling (RealESRGAN) have been added!
+## Step 3: There is no step 3!
+It's simple to get started. You don't need to install or struggle with Python, Anaconda, Docker etc.
 
-This distribution currently uses Stable Diffusion 1.4. Once the model for 1.5 becomes publicly available, the model in this distribution will be updated.
+The installer will take care of whatever is needed. A friendly [Discord community](https://discord.com/invite/u9yhsFmEkB) will help you if you face any problems.
 
-# Features in the new v2 Version:
+----
+
+# Easy for new users, powerful features for advanced users
+### Features:
 - **No Dependencies or Technical Knowledge Required**: 1-click install for Windows 10/11 and Linux. *No dependencies*, no need for WSL or Docker or Conda or technical setup. Just download and run!
-- **Face Correction (GFPGAN) and Upscaling (RealESRGAN)**
-- **In-Painting**
+- **Clutter-free UI**: a friendly and simple UI, while providing a lot of powerful features
+- Supports "*Text to Image*" and "*Image to Image*"
+- **Custom Models**: Use your own `.ckpt` file, by placing it inside the `models/stable-diffusion` folder!
 - **Live Preview**: See the image as the AI is drawing it
 - **Task Queue**: Queue up all your ideas, without waiting for the current task to finish
-- **Custom Weights**: Use your own `.ckpt` file, by placing it inside the `stable-diffusion` folder (rename it to `custom-model.ckpt`)
-- **Negative Prompt**: Specify aspects of the image to *remove*.
-- **Lots of Samplers:** ddim, plms, heun, euler, euler_a, dpm2, dpm2_a, lms
+- **In-Painting**: Specify areas of your image to paint into
+- **Face Correction (GFPGAN) and Upscaling (RealESRGAN)**
 - **Image Modifiers**: A library of *modifier tags* like *"Realistic"*, *"Pencil Sketch"*, *"ArtStation"* etc. Experiment with various styles quickly.
-- **New UI**: with cleaner design
-- **Waifu Model Support**: Just replace the `stable-diffusion\sd-v1-4.ckpt` file after installation with the Waifu model
-- Supports "*Text to Image*" and "*Image to Image*"
+- **Loopback**: Use the output image as the input image for the next img2img task
+- **Negative Prompt**: Specify aspects of the image to *remove*.
+- **Attention/Emphasis:** () in the prompt increases the model's attention to enclosed words, and [] decreases it
+- **Weighted Prompts:** Use weights for specific words in your prompt to change their importance, e.g. `red:2.4 dragon:1.2`
+- **Prompt Matrix:** (in beta) Quickly create multiple variations of your prompt, e.g. `a photograph of an astronaut riding a horse | illustration | cinematic lighting`
+- **Lots of Samplers:** ddim, plms, heun, euler, euler_a, dpm2, dpm2_a, lms
+- **Multiple Prompts File:** Queue multiple prompts by entering one prompt per line, or by running a text file
 - **NSFW Setting**: A setting in the UI to control *NSFW content*
+- **JPEG/PNG output**
+- **Save generated images to disk**
 - **Use CPU setting**: If you don't have a compatible graphics card, but still want to run it on your CPU.
 - **Auto-updater**: Gets you the latest improvements and bug-fixes to a rapidly evolving project.
 - **Low Memory Usage**: Creates 512x512 images with less than 4GB of VRAM!
+- **Developer Console**: A developer-mode for those who want to modify their Stable Diffusion code, and edit the conda environment.
 
-![Screenshot of advanced settings](media/shot-v9.jpg?raw=true)
+### Easy for new users:
+![Screenshot of the initial UI](media/shot-v10-simple.jpg?raw=true)
+
+### Powerful features for advanced users:
+![Screenshot of advanced settings](media/shot-v10.jpg?raw=true)
+
+### Live Preview
+Useful for judging (and stopping) an image quickly, without waiting for it to finish rendering.
 
-## Live Preview
 ![live-512](https://user-images.githubusercontent.com/844287/192097249-729a0a1e-a677-485e-9ccc-16a9e848fabe.gif)
 
+### Task Queue
+![Screenshot of task queue](media/task-queue-v1.jpg?raw=true)
 
 # System Requirements
 1. Windows 10/11, or Linux. Experimental support for Mac is coming soon.
-2. An NVIDIA graphics card, preferably with 4GB or more of VRAM. But if you don't have a compatible graphics card, you can still use it with a "Use CPU" setting. It'll be very slow, but it should still work.
+2. An NVIDIA graphics card, preferably with 4GB or more of VRAM. If you don't have a compatible graphics card, it'll automatically run in the slower "CPU Mode".
+3. Minimum 8 GB of RAM and 25GB of disk space.
 
-You do not need anything else. You do not need WSL, Docker or Conda. The installer will take care of it.
+You don't need to install or struggle with Python, Anaconda, Docker etc. The installer will take care of whatever is needed.
 
 # Installation
-1. **Download** [for Windows](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.16/stable-diffusion-ui-win64.zip) or [for Linux](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.16/stable-diffusion-ui-linux.tar.xz).
+1. **Download** [for Windows](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.3.5/stable-diffusion-ui-windows.zip) or [for Linux](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.3.5/stable-diffusion-ui-linux.zip).
 
 2. **Extract**:
   - For Windows: After unzipping the file, please move the `stable-diffusion-ui` folder to your `C:` (or any drive like D:, at the top root level), e.g. `C:\stable-diffusion-ui`. This will avoid a common problem with Windows (file path length limits).
@@ -57,49 +85,19 @@ This will automatically install Stable Diffusion, set it up, and start the inter
 
 **To Uninstall:** Just delete the `stable-diffusion-ui` folder to uninstall all the downloaded packages.
 
-
-# Usage
-Open http://localhost:9000 in your browser (after running step 3 previously). It may take a few moments for the back-end to be ready.
-
-## With a text description
-1. Enter a text prompt, like `a photograph of an astronaut riding a horse` in the textbox.
-2. Press `Make Image`. This will take some time, depending on your system's processing power.
-3. See the image generated using your prompt.
-
-## With an image
-1. Click `Browse..` next to `Initial Image`. Select your desired image.
-2. An optional text prompt can help you further describe the kind of image you want to generate.
-3. Press `Make Image`. See the image generated using your prompt.
-
-You can use Face Correction or Upscaling to improve the image further.
-
-**Pro tip:** You can also click `Use as Input` on a generated image, to use it as the input image for your next generation. This can be useful for sequentially refining the generated image with a single click.
-
-**Another tip:** Images with the same aspect ratio of your generated image work best. E.g. 1:1 if you're generating images sized 512x512.
-
-## Problems? Troubleshooting
-Please try the common [troubleshooting](Troubleshooting.md) steps. If that doesn't fix it, please ask on the [discord server](https://discord.com/invite/u9yhsFmEkB), or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
-
-# Image Settings
-You can also set the configuration like `seed`, `width`, `height`, `num_outputs`, `num_inference_steps` and `guidance_scale` using the 'show' button next to 'Image settings'.
-
-Use the same `seed` number to get the same image for a certain prompt. This is useful for refining a prompt without losing the basic image design. Enable the `random images` checkbox to get random images.
-
-![Screenshot of advanced settings](media/config-v6.jpg?raw=true)
-
-# System Settings
-The system settings are reachable via the cogwheel symbol on the top right. It can be used to configure whether all generated images should 
-saved be automically, or to tune the Stable Diffusion image generation.
-
-![Screenshot of advanced settings](media/system-settings-v2.jpg?raw=true)
-
-# Image Modifiers
-![Screenshot of advanced settings](media/modifiers-v1.jpg?raw=true)
+# How to use?
+Please use our [guide](https://github.com/cmdr2/stable-diffusion-ui/wiki/How-to-Use) to understand how to use the features in this UI.
 
 # Bugs reports and code contributions welcome
 If there are any problems or suggestions, please feel free to ask on the [discord server](https://discord.com/invite/u9yhsFmEkB) or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
 
-Also, please feel free to submit a pull request, if you have any code contributions in mind. Join the [discord server](https://discord.com/invite/u9yhsFmEkB) for development-related discussions, and for helping other users.
+We could really use help on these aspects (click to view tasks that need your help):
+* [User Interface](https://github.com/users/cmdr2/projects/1/views/1)
+* [Engine](https://github.com/users/cmdr2/projects/3/views/1)
+* [Installer](https://github.com/users/cmdr2/projects/4/views/1)
+* [Documentation](https://github.com/users/cmdr2/projects/5/views/1)
+
+If you have any code contributions in mind, please feel free to say Hi to us on the [discord server](https://discord.com/invite/u9yhsFmEkB). We use the Discord server for development-related discussions, and for helping users.
 
 # Disclaimer
 The authors of this project are not responsible for any content generated using this interface.

Troubleshooting.md

@@ -1,75 +1 @@
-Common issues and their solutions. If these solutions don't work, please feel free to ask at the [discord server](https://discord.com/invite/u9yhsFmEkB) or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
-
-## RuntimeError: CUDA out of memory
-This can happen if your PC has less than 6GB of VRAM.
-
-Try disabling the "Turbo mode" setting under "Advanced Settings", since that takes an additional 1 GB of VRAM (to increase the speed).
-
-Additionally, a common reason for this error is that you're using an initial image larger than 768x768 pixels. Try using a smaller initial image.
-
-Also try generating smaller sized images.
-
-## basicsr module not found
-For Windows: Please download and extract basicsr from [here](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.16/basicsr-win64.zip), and place the `basicsr` folder inside the `stable-diffusion-ui\stable-diffusion\env\lib\site-packages` folder. Then run the `Start Stable Diffusion UI.cmd` file again.
-
-For Linux: Please contact on the [discord server](https://discord.com/invite/u9yhsFmEkB).
-
-## No ldm found, or antlr4 or any other missing module, or ClobberError: This transaction has incompatible packages due to a shared path
-On Windows, please ensure that you had placed the `stable-diffusion-ui` folder after unzipping to the root of C: or D: (or any drive). For e.g. `C:\stable-diffusion-ui`. **Note:** This has to be done **before** you start the installation process. If you have already installed (and are facing this error), please delete the installed folder, and start fresh by unzipping and placing the folder at the top of your drive.
-
-This error can also be caused if you already have conda/miniconda/anaconda installed, due to package conflicts. Please open your Anaconda Prompt, and run `conda clean --all` to clean up unused packages.
-
-If nothing works, this could be due to a corrupted installation. Please try reinstalling this, by deleting the installed folder, and unzipping from the downloaded zip file.
-
-## Killed uvicorn server:app --app-dir ... --port 9000 --host 0.0.0.0
-This happens if your PC ran out of RAM. Stable Diffusion requires a lot of RAM, and requires atleast 10 GB of RAM to work well. You can also try closing all other applications before running Stable Diffusion UI.
-
-## Green image generated
-This usually happens if you're running NVIDIA 1650 or 1660 Super. To solve this, please close and run the Stable Diffusion command on your computer. If you're using the older Docker-based solution (v1), please upgrade to v2: https://github.com/cmdr2/stable-diffusion-ui/tree/v2#installation
-
-If you're still seeing this error, please try enabling "Full Precision" under "Advanced Settings" in the Stable Diffusion UI.
-
-## './docker-compose.yml' is invalid:
-> ERROR: The Compose file './docker-compose.yml' is invalid because:
-> services.stability-ai.deploy.resources.reservations value Additional properties are not allowed ('devices' was unexpected)
-
-Please ensure you have `docker-compose` version 1.29 or higher. Check `docker-compose --version`, and if required [update it to 1.29](https://docs.docker.com/compose/install/). (Thanks [HVRyan](https://github.com/HVRyan))
-
-## RuntimeError: Found no NVIDIA driver on your system:
-If you have an NVIDIA GPU and the latest [NVIDIA driver](http://www.nvidia.com/Download/index.aspx), please ensure that you've installed [nvidia-container-toolkit](https://stackoverflow.com/a/58432877). (Thanks [u/exintrovert420](https://www.reddit.com/user/exintrovert420/))
-
-## Some other process is already running at port 9000 / port 9000 could not be bound
-You can override the port used. Please change `docker-compose.yml` inside the project directory, and update the line `9000:9000` to `1337:9000` (where 1337 is whichever port number you want).
-
-After doing this, please restart your server, by running `./server restart`.
-
-After this, you can access the server at `http://localhost:1337` (where 1337 is the new port you specified earlier).
-
-## RuntimeError: CUDA error: unknown error
-Please ensure that you have an NVIDIA GPU and the latest [NVIDIA driver](http://www.nvidia.com/Download/index.aspx), and that you've installed [nvidia-container-toolkit](https://stackoverflow.com/a/58432877).
-
-Also, if you are using WSL (Windows), please ensure you have the latest WSL kernel by running `wsl --shutdown` and then `wsl --update`. (Thanks [AndrWeisR](https://github.com/AndrWeisR))
-
-# For support queries
-## Entering a conda environment in an existing installation
-This will give you an activated conda environment in the terminal, so you can run commands and force-install any packages, if required.
-
-Users don't need to have the Anaconda Prompt installed to do this anymore, since the installer bundles a portable version of conda inside it. Just follow these steps.
-
-**Windows:**
-1. Open the terminal: Press Win+R, type "cmd", and press "Run"
-2. Type `cd C:\stable-diffusion-ui` and press enter (or wherever you've installed it)
-3. Type `installer\Scripts\activate.bat` and press enter
-4. Type `cd stable-diffusion` and press enter
-5. Type `conda activate .\env` and press enter
-6. Type `python --version` and press enter. You should see 3.8.5.
-
-**Linux:**
-1. Open the terminal
-2. Type `cd /path/to/stable-diffusion-ui` and press enter
-3. Type `installer/bin/activate` and press enter
-4. Type `cd stable-diffusion` and press enter
-5. Type `conda activate ./env` and press enter
-6. Type `python --version` and press enter. You should see 3.8.5.
-
-This will give you an activated conda environment. To confirm, type `python --version` and press enter. You should see 3.8.5.
+Moved to https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting

build.bat

@@ -8,40 +8,40 @@
 set /p answer=Are you a developer of this project (Y/N)?
 if /i "%answer:~,1%" NEQ "Y" exit /b
 
-@set PYTHONNOUSERSITE=1
+mkdir dist\win\stable-diffusion-ui\scripts
+@REM mkdir dist\linux-mac\stable-diffusion-ui\scripts
 
-@mkdir dist\stable-diffusion-ui
+@rem copy the installer files for Windows
 
-@echo "Downloading components for the installer.."
+copy scripts\on_env_start.bat dist\win\stable-diffusion-ui\scripts\
+copy scripts\bootstrap.bat dist\win\stable-diffusion-ui\scripts\
+copy "scripts\Start Stable Diffusion UI.cmd" dist\win\stable-diffusion-ui\
+copy LICENSE dist\win\stable-diffusion-ui\
+copy "CreativeML Open RAIL-M License" dist\win\stable-diffusion-ui\
+copy "How to install and run.txt" dist\win\stable-diffusion-ui\
+echo. > dist\win\stable-diffusion-ui\scripts\install_status.txt
 
-@call conda env create --prefix installer -f environment.yaml
-@call conda activate .\installer
+@rem copy the installer files for Linux and Mac
 
-@echo "Creating a distributable package.."
+@REM copy scripts\on_env_start.sh dist\linux-mac\stable-diffusion-ui\scripts\
+@REM copy scripts\bootstrap.sh dist\linux-mac\stable-diffusion-ui\scripts\
+@REM copy scripts\start.sh dist\linux-mac\stable-diffusion-ui\
+@REM copy LICENSE dist\linux-mac\stable-diffusion-ui\
+@REM copy "CreativeML Open RAIL-M License" dist\linux-mac\stable-diffusion-ui\
+@REM copy "How to install and run.txt" dist\linux-mac\stable-diffusion-ui\
+@REM echo. > dist\linux-mac\stable-diffusion-ui\scripts\install_status.txt
 
-@call conda install -c conda-forge -y conda-pack
-@call conda pack --n-threads -1 --prefix installer --format tar
+@rem make the zip
 
-@cd dist\stable-diffusion-ui
-@mkdir installer
+cd dist\win
+call powershell Compress-Archive -Path stable-diffusion-ui -DestinationPath ..\stable-diffusion-ui-windows.zip
+cd ..\..
 
-@call tar -xf ..\..\installer.tar -C installer
+@REM cd dist\linux-mac
+@REM call powershell Compress-Archive -Path stable-diffusion-ui -DestinationPath ..\stable-diffusion-ui-linux.zip
+@REM call powershell Compress-Archive -Path stable-diffusion-ui -DestinationPath ..\stable-diffusion-ui-mac.zip
+@REM cd ..\..
 
-@mkdir scripts
+echo "Build ready. Upload the zip files inside the 'dist' folder."
 
-@copy ..\..\scripts\on_env_start.bat scripts\
-@copy "..\..\scripts\Start Stable Diffusion UI.cmd" .
-@copy ..\..\LICENSE .
-@copy "..\..\CreativeML Open RAIL-M License" .
-@copy "..\..\How to install and run.txt" .
-@echo. > scripts\install_status.txt
-
-@echo "Build ready. Zip the 'dist\stable-diffusion-ui' folder."
-
-@echo "Cleaning up.."
-
-@cd ..\..
-
-@rmdir /s /q installer
-
-@del installer.tar
+pause

build.sh

@@ -11,45 +11,39 @@ case $yn in
     * ) exit;;
 esac
 
-export PYTHONNOUSERSITE=1
+# mkdir -p dist/win/stable-diffusion-ui/scripts
+mkdir -p dist/linux-mac/stable-diffusion-ui/scripts
 
-mkdir -p dist/stable-diffusion-ui
+# copy the installer files for Windows
 
-echo "Downloading components for the installer.."
+# cp scripts/on_env_start.bat dist/win/stable-diffusion-ui/scripts/
+# cp scripts/bootstrap.bat dist/win/stable-diffusion-ui/scripts/
+# cp "scripts/Start Stable Diffusion UI.cmd" dist/win/stable-diffusion-ui/
+# cp LICENSE dist/win/stable-diffusion-ui/
+# cp "CreativeML Open RAIL-M License" dist/win/stable-diffusion-ui/
+# cp "How to install and run.txt" dist/win/stable-diffusion-ui/
+# echo "" > dist/win/stable-diffusion-ui/scripts/install_status.txt
 
-source ~/miniconda3/etc/profile.d/conda.sh
+# copy the installer files for Linux and Mac
 
-conda install -c conda-forge -y conda-pack
+cp scripts/on_env_start.sh dist/linux-mac/stable-diffusion-ui/scripts/
+cp scripts/bootstrap.sh dist/linux-mac/stable-diffusion-ui/scripts/
+cp scripts/functions.sh dist/linux-mac/stable-diffusion-ui/scripts/
+cp scripts/start.sh dist/linux-mac/stable-diffusion-ui/
+cp LICENSE dist/linux-mac/stable-diffusion-ui/
+cp "CreativeML Open RAIL-M License" dist/linux-mac/stable-diffusion-ui/
+cp "How to install and run.txt" dist/linux-mac/stable-diffusion-ui/
+echo "" > dist/linux-mac/stable-diffusion-ui/scripts/install_status.txt
 
-conda env create --prefix installer -f environment.yaml
-conda activate ./installer
+# make the zip
 
-echo "Creating a distributable package.."
-
-conda pack --n-threads -1 --prefix installer --format tar
-
-cd dist/stable-diffusion-ui
-mkdir installer
-
-tar -xf ../../installer.tar -C installer
-
-mkdir scripts
-
-cp ../../scripts/on_env_start.sh scripts/
-cp ../../scripts/start.sh .
-cp ../../LICENSE .
-cp "../../CreativeML Open RAIL-M License" .
-cp "../../How to install and run.txt" .
-echo "" > scripts/install_status.txt
-
-chmod u+x start.sh
-
-echo "Build ready. Zip the 'dist/stable-diffusion-ui' folder."
-
-echo "Cleaning up.."
+# cd dist/win
+# zip -r ../stable-diffusion-ui-windows.zip stable-diffusion-ui
+# cd ../..
 
+cd dist/linux-mac
+zip -r ../stable-diffusion-ui-linux.zip stable-diffusion-ui
+zip -r ../stable-diffusion-ui-mac.zip stable-diffusion-ui
 cd ../..
 
-rm -rf installer
-
-rm installer.tar
+echo "Build ready. Upload the zip files inside the 'dist' folder."

scripts/Developer Console.cmd

@@ -2,13 +2,33 @@
 
 echo "Opening Stable Diffusion UI - Developer Console.." & echo.
 
-@call installer\Scripts\activate.bat
+set PATH=C:\Windows\System32;%PATH%
 
-@call conda-unpack
+@rem set legacy and new installer's PATH, if they exist
+if exist "installer" set PATH=%cd%\installer;%cd%\installer\Library\bin;%cd%\installer\Scripts;%cd%\installer\Library\usr\bin;%PATH%
+if exist "installer_files\env" set PATH=%cd%\installer_files\env;%cd%\installer_files\env\Library\bin;%cd%\installer_files\env\Scripts;%cd%\installer_files\Library\usr\bin;%PATH%
 
-@call conda --version
-@call git --version
+set PYTHONPATH=%cd%\installer;%cd%\installer_files\env
 
-@call conda activate .\stable-diffusion\env
+@rem activate the installer env
+call conda activate
 
-cmd /k
+@rem Test the environment
+echo "Environment Info:"
+call where git
+call git --version
+
+call where conda
+call conda --version
+
+echo.
+
+@rem activate the environment
+call conda activate .\stable-diffusion\env
+
+call where python
+call python --version
+
+echo.
+
+cmd /k

scripts/Start Stable Diffusion UI.cmd

@@ -1,19 +1,27 @@
 @echo off
 
-@REM Delete the post-activate hook from the old installer
-if exist "installer\etc\conda\activate.d\post_activate.bat" (
-    echo. > installer\etc\conda\activate.d\post_activate.bat
-)
+cd /d %~dp0
+set PATH=C:\Windows\System32;%PATH%
 
-@call installer\Scripts\activate.bat
+@rem set legacy installer's PATH, if it exists
+if exist "installer" set PATH=%cd%\installer;%cd%\installer\Library\bin;%cd%\installer\Scripts;%cd%\installer\Library\usr\bin;%PATH%
 
-@call conda-unpack
+@rem Setup the packages required for the installer
+call scripts\bootstrap.bat
 
-@call conda --version
-@call git --version
+@rem set new installer's PATH, if it downloaded any packages
+if exist "installer_files\env" set PATH=%cd%\installer_files\env;%cd%\installer_files\env\Library\bin;%cd%\installer_files\env\Scripts;%cd%\installer_files\Library\usr\bin;%PATH%
 
-@cd installer
+set PYTHONPATH=%cd%\installer;%cd%\installer_files\env
 
-@call ..\scripts\on_env_start.bat
+@rem Test the bootstrap
+call where git
+call git --version
+
+call where conda
+call conda --version
+
+@rem Download the rest of the installer and UI
+call scripts\on_env_start.bat
 
 @pause

scripts/bootstrap.bat

@@ -0,0 +1,77 @@
+@echo off
+
+@rem This script will install git and conda (if not found on the PATH variable)
+@rem  using micromamba (an 8mb static-linked single-file binary, conda replacement).
+@rem For users who already have git and conda, this step will be skipped.
+
+@rem This enables a user to install this project without manually installing conda and git.
+
+@rem config
+set MAMBA_ROOT_PREFIX=%cd%\installer_files\mamba
+set INSTALL_ENV_DIR=%cd%\installer_files\env
+set LEGACY_INSTALL_ENV_DIR=%cd%\installer
+set MICROMAMBA_DOWNLOAD_URL=https://github.com/cmdr2/stable-diffusion-ui/releases/download/v1.1/micromamba.exe
+set umamba_exists=F
+
+set OLD_APPDATA=%APPDATA%
+set OLD_USERPROFILE=%USERPROFILE%
+set APPDATA=%cd%\installer_files\appdata
+set USERPROFILE=%cd%\profile
+
+@rem figure out whether git and conda needs to be installed
+if exist "%INSTALL_ENV_DIR%" set PATH=%INSTALL_ENV_DIR%;%INSTALL_ENV_DIR%\Library\bin;%INSTALL_ENV_DIR%\Scripts;%INSTALL_ENV_DIR%\Library\usr\bin;%PATH%
+
+set PACKAGES_TO_INSTALL=
+
+if not exist "%LEGACY_INSTALL_ENV_DIR%\etc\profile.d\conda.sh" (
+    if not exist "%INSTALL_ENV_DIR%\etc\profile.d\conda.sh" set PACKAGES_TO_INSTALL=%PACKAGES_TO_INSTALL% conda
+)
+
+call git --version >.tmp1 2>.tmp2
+if "%ERRORLEVEL%" NEQ "0" set PACKAGES_TO_INSTALL=%PACKAGES_TO_INSTALL% git
+
+call "%MAMBA_ROOT_PREFIX%\micromamba.exe" --version >.tmp1 2>.tmp2
+if "%ERRORLEVEL%" EQU "0" set umamba_exists=T
+
+@rem (if necessary) install git and conda into a contained environment
+if "%PACKAGES_TO_INSTALL%" NEQ "" (
+    @rem download micromamba
+    if "%umamba_exists%" == "F" (
+        echo "Downloading micromamba from %MICROMAMBA_DOWNLOAD_URL% to %MAMBA_ROOT_PREFIX%\micromamba.exe"
+
+        mkdir "%MAMBA_ROOT_PREFIX%"
+        call curl -Lk "%MICROMAMBA_DOWNLOAD_URL%" > "%MAMBA_ROOT_PREFIX%\micromamba.exe"
+
+        @REM if "%ERRORLEVEL%" NEQ "0" (
+        @REM     echo "There was a problem downloading micromamba. Cannot continue."
+        @REM     pause
+        @REM     exit /b
+        @REM )
+
+        mkdir "%APPDATA%"
+        mkdir "%USERPROFILE%"
+
+        @rem test the mamba binary
+        echo Micromamba version:
+        call "%MAMBA_ROOT_PREFIX%\micromamba.exe" --version
+    )
+
+    @rem create the installer env
+    if not exist "%INSTALL_ENV_DIR%" (
+        call "%MAMBA_ROOT_PREFIX%\micromamba.exe" create -y --prefix "%INSTALL_ENV_DIR%"
+    )
+
+    echo "Packages to install:%PACKAGES_TO_INSTALL%"
+
+    call "%MAMBA_ROOT_PREFIX%\micromamba.exe" install -y --prefix "%INSTALL_ENV_DIR%" -c conda-forge %PACKAGES_TO_INSTALL%
+
+    if not exist "%INSTALL_ENV_DIR%" (
+        echo "There was a problem while installing%PACKAGES_TO_INSTALL% using micromamba. Cannot continue."
+        pause
+        exit /b
+    )
+)
+
+@rem revert to the old APPDATA. only needed it for bypassing a bug in micromamba (with special characters)
+set APPDATA=%OLD_APPDATA%
+set USERPROFILE=%OLD_USERPROFILE%

scripts/bootstrap.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# This script will install git and conda (if not found on the PATH variable)
+#  using micromamba (an 8mb static-linked single-file binary, conda replacement).
+# For users who already have git and conda, this step will be skipped.
+
+# This enables a user to install this project without manually installing conda and git.
+
+source ./scripts/functions.sh
+
+set -o pipefail
+
+OS_NAME=$(uname -s)
+case "${OS_NAME}" in
+    Linux*)     OS_NAME="linux";;
+    Darwin*)    OS_NAME="osx";;
+    *)          echo "Unknown OS: $OS_NAME! This script runs only on Linux or Mac" && exit
+esac
+
+OS_ARCH=$(uname -m)
+case "${OS_ARCH}" in
+    x86_64*)    OS_ARCH="64";;
+    arm64*)     OS_ARCH="arm64";;
+    *)          echo "Unknown system architecture: $OS_ARCH! This script runs only on x86_64 or arm64" && exit
+esac
+
+# https://mamba.readthedocs.io/en/latest/installation.html
+if [ "$OS_NAME" == "linux" ] && [ "$OS_ARCH" == "arm64" ]; then OS_ARCH="aarch64"; fi
+
+# config
+export MAMBA_ROOT_PREFIX="$(pwd)/installer_files/mamba"
+INSTALL_ENV_DIR="$(pwd)/installer_files/env"
+LEGACY_INSTALL_ENV_DIR="$(pwd)/installer"
+MICROMAMBA_DOWNLOAD_URL="https://micro.mamba.pm/api/micromamba/${OS_NAME}-${OS_ARCH}/latest"
+umamba_exists="F"
+
+# figure out whether git and conda needs to be installed
+if [ -e "$INSTALL_ENV_DIR" ]; then export PATH="$INSTALL_ENV_DIR/bin:$PATH"; fi
+
+PACKAGES_TO_INSTALL=""
+
+if [ ! -e "$LEGACY_INSTALL_ENV_DIR/etc/profile.d/conda.sh" ] && [ ! -e "$INSTALL_ENV_DIR/etc/profile.d/conda.sh" ]; then PACKAGES_TO_INSTALL="$PACKAGES_TO_INSTALL conda"; fi
+if ! hash "git" &>/dev/null; then PACKAGES_TO_INSTALL="$PACKAGES_TO_INSTALL git"; fi
+
+if "$MAMBA_ROOT_PREFIX/micromamba" --version &>/dev/null; then umamba_exists="T"; fi
+
+# (if necessary) install git and conda into a contained environment
+if [ "$PACKAGES_TO_INSTALL" != "" ]; then
+    # download micromamba
+    if [ "$umamba_exists" == "F" ]; then
+        echo "Downloading micromamba from $MICROMAMBA_DOWNLOAD_URL to $MAMBA_ROOT_PREFIX/micromamba"
+
+        mkdir -p "$MAMBA_ROOT_PREFIX"
+        curl -L "$MICROMAMBA_DOWNLOAD_URL" | tar -xvj bin/micromamba -O > "$MAMBA_ROOT_PREFIX/micromamba"
+
+        if [ "$?" != "0" ]; then
+            echo
+            echo "EE micromamba download failed"
+            echo "EE If the lines above contain 'bzip2: Cannot exec', your system doesn't have bzip2 installed"
+            echo "EE If there are network errors, please check your internet setup"
+            fail "micromamba download failed"
+        fi
+
+        chmod u+x "$MAMBA_ROOT_PREFIX/micromamba"
+
+        # test the mamba binary
+        echo "Micromamba version:"
+        "$MAMBA_ROOT_PREFIX/micromamba" --version
+    fi
+
+    # create the installer env
+    if [ ! -e "$INSTALL_ENV_DIR" ]; then
+        "$MAMBA_ROOT_PREFIX/micromamba" create -y --prefix "$INSTALL_ENV_DIR" || fail "unable to create the install environment"
+    fi
+   
+    if [ ! -e "$INSTALL_ENV_DIR" ]; then
+        fail "There was a problem while installing$PACKAGES_TO_INSTALL using micromamba. Cannot continue."
+    fi
+
+    echo "Packages to install:$PACKAGES_TO_INSTALL"
+
+    "$MAMBA_ROOT_PREFIX/micromamba" install -y --prefix "$INSTALL_ENV_DIR" -c conda-forge $PACKAGES_TO_INSTALL
+    if [ "$?" != "0" ]; then
+        fail "Installation of the packages '$PACKAGES_TO_INSTALL' failed."
+    fi
+fi

scripts/developer_console.sh

@@ -1,17 +1,42 @@
 #!/bin/bash
 
+cd "$(dirname "${BASH_SOURCE[0]}")"
+
 if [ "$0" == "bash" ]; then
   echo "Opening Stable Diffusion UI - Developer Console.."
   echo ""
 
-  source installer/bin/activate
+  # set legacy and new installer's PATH, if they exist
+  if [ -e "installer" ]; then export PATH="$(pwd)/installer/bin:$PATH"; fi
+  if [ -e "installer_files/env" ]; then export PATH="$(pwd)/installer_files/env/bin:$PATH"; fi
 
-  conda-unpack
+  # activate the installer env
+  CONDA_BASEPATH=$(conda info --base)
+  source "$CONDA_BASEPATH/etc/profile.d/conda.sh" # avoids the 'shell not initialized' error
 
-  conda --version
+  conda activate
+
+  # test the environment
+  echo "Environment Info:"
+  which git
   git --version
 
+  which conda
+  conda --version
+
+  echo ""
+
+  # activate the environment
+  CONDA_BASEPATH=$(conda info --base)
+  source "$CONDA_BASEPATH/etc/profile.d/conda.sh" # otherwise conda complains about 'shell not initialized' (needed when running in a script)
+
   conda activate ./stable-diffusion/env
+
+  which python
+  python --version
+
+  echo ""
 else
-  bash --init-file open_dev_console.sh
-fi
+  file_name=$(basename "${BASH_SOURCE[0]}")
+  bash --init-file "$file_name"
+fi

scripts/functions.sh

@@ -0,0 +1,32 @@
+#
+# utility functions for all scripts
+#
+
+fail() {
+    echo
+    echo "EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE"
+    echo
+    if [ "$1" != "" ]; then
+        echo ERROR: $1
+    else
+        echo An error occurred.
+    fi
+    cat <<EOF
+
+Error downloading Stable Diffusion UI. Sorry about that, please try to:
+ 1. Run this installer again.
+ 2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting
+ 3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB
+ 4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues
+
+Thanks!
+
+
+EOF
+    read -p "Press any key to continue"
+    exit 1
+
+}
+
+
+

scripts/on_env_start.bat

@@ -4,8 +4,6 @@
 
 set PATH=C:\Windows\System32;%PATH%
 
-@cd ..
-
 if exist "scripts\config.bat" (
     @call scripts\config.bat
 )
@@ -14,7 +12,7 @@ if "%update_branch%"=="" (
     set update_branch=main
 )
 
-@>nul grep -c "conda_sd_ui_deps_installed" scripts\install_status.txt
+@>nul findstr /m "conda_sd_ui_deps_installed" scripts\install_status.txt
 @if "%ERRORLEVEL%" NEQ "0" (
     for /f "tokens=*" %%a in ('python -c "import os; parts = os.getcwd().split(os.path.sep); print(len(parts))"') do if "%%a" NEQ "2" (
         echo. & echo "!!!! WARNING !!!!" & echo.
@@ -28,14 +26,14 @@ if "%update_branch%"=="" (
     )
 )
 
-@>nul grep -c "sd_ui_git_cloned" scripts\install_status.txt
+@>nul findstr /m "sd_ui_git_cloned" scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Stable Diffusion UI's git repository was already installed. Updating from %update_branch%.."
 
     @cd sd-ui-files
 
     @call git reset --hard
-    @call git checkout "%update_branch%"
+    @call git -c advice.detachedHead=false checkout "%update_branch%"
     @call git pull
 
     @cd ..
@@ -46,17 +44,18 @@ if "%update_branch%"=="" (
     @call git clone -b "%update_branch%" https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files && (
         @echo sd_ui_git_cloned >> scripts\install_status.txt
     ) || (
-        @echo "Error downloading Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
+        @echo "Error downloading Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         @exit /b
     )
 )
 
-@xcopy sd-ui-files\ui ui /s /i /Y
+@xcopy sd-ui-files\ui ui /s /i /Y /q
 @copy sd-ui-files\scripts\on_sd_start.bat scripts\ /Y
+@copy sd-ui-files\scripts\bootstrap.bat scripts\ /Y
 @copy "sd-ui-files\scripts\Start Stable Diffusion UI.cmd" . /Y
 @copy "sd-ui-files\scripts\Developer Console.cmd" . /Y
 
 @call scripts\on_sd_start.bat
 
-@pause
+@pause

scripts/on_env_start.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+source ./scripts/functions.sh
+
 printf "\n\nStable Diffusion UI\n\n"
 
 if [ -f "scripts/config.sh" ]; then
@@ -16,7 +18,7 @@ if [ -f "scripts/install_status.txt" ] && [ `grep -c sd_ui_git_cloned scripts/in
     cd sd-ui-files
 
     git reset --hard
-    git checkout "$update_branch"
+    git -c advice.detachedHead=false checkout "$update_branch"
     git pull
 
     cd ..
@@ -27,15 +29,14 @@ else
     if git clone -b "$update_branch" https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files ; then
         echo sd_ui_git_cloned >> scripts/install_status.txt
     else
-        printf "\n\nError downloading Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "git clone failed"
     fi
 fi
 
 rm -rf ui
 cp -Rf sd-ui-files/ui .
 cp sd-ui-files/scripts/on_sd_start.sh scripts/
+cp sd-ui-files/scripts/bootstrap.sh scripts/
 cp sd-ui-files/scripts/start.sh .
 cp sd-ui-files/scripts/developer_console.sh .
 

scripts/on_sd_start.bat

@@ -1,16 +1,24 @@
 @echo off
 
 @copy sd-ui-files\scripts\on_env_start.bat scripts\ /Y
+@copy sd-ui-files\scripts\bootstrap.bat scripts\ /Y
+
+if exist "%cd%\profile" (
+    set USERPROFILE=%cd%\profile
+)
+
+@rem activate the installer env
+call conda activate
 
 @REM Caution, this file will make your eyes and brain bleed. It's such an unholy mess.
 @REM Note to self: Please rewrite this in Python. For the sake of your own sanity.
 
-@copy "sd-ui-files\scripts\Developer Console.cmd" . /Y
+@REM remove the old version of the dev console script, if it's still present
 if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
 
 @call python -c "import os; import shutil; frm = 'sd-ui-files\\ui\\hotfix\\9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'; dst = os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'transformers', '9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'); shutil.copyfile(frm, dst) if os.path.exists(dst) else print(''); print('Hotfixed broken JSON file from OpenAI');"
 
-@>nul grep -c "sd_git_cloned" scripts\install_status.txt
+@>nul findstr /m "sd_git_cloned" scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Stable Diffusion's git repository was already installed. Updating.."
 
@@ -18,7 +26,7 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
 
     @call git reset --hard
     @call git pull
-    @call git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+    @call git -c advice.detachedHead=false checkout f6cfebffa752ee11a7b07497b8529d5971de916c
 
     @call git apply ..\ui\sd_internal\ddim_callback.patch
     @call git apply ..\ui\sd_internal\env_yaml.patch
@@ -30,13 +38,13 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
     @call git clone https://github.com/basujindal/stable-diffusion.git && (
         @echo sd_git_cloned >> scripts\install_status.txt
     ) || (
-        @echo "Error downloading Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
+        @echo "Error downloading Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         @exit /b
     )
 
     @cd stable-diffusion
-    @call git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+    @call git -c advice.detachedHead=false checkout f6cfebffa752ee11a7b07497b8529d5971de916c
 
     @call git apply ..\ui\sd_internal\ddim_callback.patch
     @call git apply ..\ui\sd_internal\env_yaml.patch
@@ -46,7 +54,7 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
 
 @cd stable-diffusion
 
-@>nul grep -c "conda_sd_env_created" ..\scripts\install_status.txt
+@>nul findstr /m "conda_sd_env_created" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Packages necessary for Stable Diffusion were already installed"
 
@@ -59,8 +67,14 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
     @REM prevent conda from using packages from the user's home directory, to avoid conflicts
     @set PYTHONNOUSERSITE=1
 
+    set USERPROFILE=%cd%\profile
+    set TMP=%cd%\tmp
+    set TEMP=%cd%\tmp
+
+    set PYTHONPATH=%cd%;%cd%\env\lib\site-packages
+
     @call conda env create --prefix env -f environment.yaml || (
-        @echo. & echo "Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -68,13 +82,13 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
     @call conda activate .\env
 
     @call conda install -c conda-forge -y --prefix env antlr4-python3-runtime=4.8 || (
-        @echo. & echo "Error installing antlr4-python3-runtime for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error installing antlr4-python3-runtime for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
 
     for /f "tokens=*" %%a in ('python -c "import torch; import ldm; import transformers; import numpy; import antlr4; print(42)"') do if "%%a" NEQ "42" (
-        @echo. & echo "Dependency test failed! Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -84,7 +98,7 @@ if exist "Open Developer Console.cmd" del "Open Developer Console.cmd"
 
 set PATH=C:\Windows\System32;%PATH%
 
-@>nul grep -c "conda_sd_gfpgan_deps_installed" ..\scripts\install_status.txt
+@>nul findstr /m "conda_sd_gfpgan_deps_installed" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Packages necessary for GFPGAN (Face Correction) were already installed"
 ) else (
@@ -92,20 +106,26 @@ set PATH=C:\Windows\System32;%PATH%
 
     @set PYTHONNOUSERSITE=1
 
+    set USERPROFILE=%cd%\profile
+    set TMP=%cd%\tmp
+    set TEMP=%cd%\tmp
+
+    set PYTHONPATH=%cd%;%cd%\env\lib\site-packages
+
     @call pip install -e git+https://github.com/TencentARC/GFPGAN#egg=GFPGAN || (
-        @echo. & echo "Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
 
     @call pip install basicsr==1.4.2 || (
-        @echo. & echo "Error installing the basicsr package necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error installing the basicsr package necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
 
     for /f "tokens=*" %%a in ('python -c "from gfpgan import GFPGANer; print(42)"') do if "%%a" NEQ "42" (
-        @echo. & echo "Dependency test failed! Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -113,7 +133,7 @@ set PATH=C:\Windows\System32;%PATH%
     @echo conda_sd_gfpgan_deps_installed >> ..\scripts\install_status.txt
 )
 
-@>nul grep -c "conda_sd_esrgan_deps_installed" ..\scripts\install_status.txt
+@>nul findstr /m "conda_sd_esrgan_deps_installed" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Packages necessary for ESRGAN (Resolution Upscaling) were already installed"
 ) else (
@@ -121,14 +141,20 @@ set PATH=C:\Windows\System32;%PATH%
 
     @set PYTHONNOUSERSITE=1
 
+    set USERPROFILE=%cd%\profile
+    set TMP=%cd%\tmp
+    set TEMP=%cd%\tmp
+
+    set PYTHONPATH=%cd%;%cd%\env\lib\site-packages
+
     @call pip install -e git+https://github.com/xinntao/Real-ESRGAN#egg=realesrgan || (
-        @echo. & echo "Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
 
     for /f "tokens=*" %%a in ('python -c "from basicsr.archs.rrdbnet_arch import RRDBNet; from realesrgan import RealESRGANer; print(42)"') do if "%%a" NEQ "42" (
-        @echo. & echo "Dependency test failed! Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -136,7 +162,7 @@ set PATH=C:\Windows\System32;%PATH%
     @echo conda_sd_esrgan_deps_installed >> ..\scripts\install_status.txt
 )
 
-@>nul grep -c "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
+@>nul findstr /m "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     echo "Packages necessary for Stable Diffusion UI were already installed"
 ) else (
@@ -144,22 +170,28 @@ set PATH=C:\Windows\System32;%PATH%
 
     @set PYTHONNOUSERSITE=1
 
+    set USERPROFILE=%cd%\profile
+    set TMP=%cd%\tmp
+    set TEMP=%cd%\tmp
+
+    set PYTHONPATH=%cd%;%cd%\env\lib\site-packages
+
     @call conda install -c conda-forge -y --prefix env uvicorn fastapi || (
-        echo "Error installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
+        echo "Error installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         exit /b
     )
 )
 
 call WHERE uvicorn > .tmp
-@>nul grep -c "uvicorn" .tmp
+@>nul findstr /m "uvicorn" .tmp
 @if "%ERRORLEVEL%" NEQ "0" (
-    @echo. & echo "UI packages not found! Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+    @echo. & echo "UI packages not found! Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
     pause
     exit /b
 )
 
-@>nul grep -c "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
+@>nul findstr /m "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" NEQ "0" (
     @echo conda_sd_ui_deps_installed >> ..\scripts\install_status.txt
 )
@@ -167,7 +199,9 @@ call WHERE uvicorn > .tmp
 
 
 if not exist "..\models\stable-diffusion" mkdir "..\models\stable-diffusion"
+if not exist "..\models\vae" mkdir "..\models\vae"
 echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
+echo. > "..\models\vae\Put your VAE files here.txt"
 
 @if exist "sd-v1-4.ckpt" (
     for %%I in ("sd-v1-4.ckpt") do if "%%~zI" EQU "4265380512" (
@@ -194,12 +228,12 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
     @if exist "sd-v1-4.ckpt" (
         for %%I in ("sd-v1-4.ckpt") do if "%%~zI" NEQ "4265380512" (
             echo. & echo "Error: The downloaded model file was invalid! Bytes downloaded: %%~zI" & echo.
-            echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
             pause
             exit /b
         )
     ) else (
-        @echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -224,12 +258,12 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
     @if exist "GFPGANv1.3.pth" (
         for %%I in ("GFPGANv1.3.pth") do if "%%~zI" NEQ "348632874" (
             echo. & echo "Error: The downloaded GFPGAN model file was invalid! Bytes downloaded: %%~zI" & echo.
-            echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
             pause
             exit /b
         )
     ) else (
-        @echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -254,12 +288,12 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
     @if exist "RealESRGAN_x4plus.pth" (
         for %%I in ("RealESRGAN_x4plus.pth") do if "%%~zI" NEQ "67040989" (
             echo. & echo "Error: The downloaded ESRGAN x4plus model file was invalid! Bytes downloaded: %%~zI" & echo.
-            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
             pause
             exit /b
         )
     ) else (
-        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -284,12 +318,12 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
     @if exist "RealESRGAN_x4plus_anime_6B.pth" (
         for %%I in ("RealESRGAN_x4plus_anime_6B.pth") do if "%%~zI" NEQ "17938799" (
             echo. & echo "Error: The downloaded ESRGAN x4plus_anime model file was invalid! Bytes downloaded: %%~zI" & echo.
-            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
             pause
             exit /b
         )
     ) else (
-        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
@@ -297,7 +331,37 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
 
 
 
-@>nul grep -c "sd_install_complete" ..\scripts\install_status.txt
+@if exist "..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt" (
+    for %%I in ("..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt") do if "%%~zI" EQU "334695179" (
+        echo "Data files (weights) necessary for the default VAE (sd-vae-ft-mse-original) were already downloaded"
+    ) else (
+        echo. & echo "The default VAE (sd-vae-ft-mse-original) file present at models\vae\vae-ft-mse-840000-ema-pruned.ckpt is invalid. It is only %%~zI bytes in size. Re-downloading.." & echo.
+        del "..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt"
+    )
+)
+
+@if not exist "..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt" (
+    @echo. & echo "Downloading data files (weights) for the default VAE (sd-vae-ft-mse-original).." & echo.
+
+    @call curl -L -k https://huggingface.co/stabilityai/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt > ..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt
+
+    @if exist "..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt" (
+        for %%I in ("..\models\vae\vae-ft-mse-840000-ema-pruned.ckpt") do if "%%~zI" NEQ "334695179" (
+            echo. & echo "Error: The downloaded default VAE (sd-vae-ft-mse-original) file was invalid! Bytes downloaded: %%~zI" & echo.
+            echo. & echo "Error downloading the data files (weights) for the default VAE (sd-vae-ft-mse-original). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            pause
+            exit /b
+        )
+    ) else (
+        @echo. & echo "Error downloading the data files (weights) for the default VAE (sd-vae-ft-mse-original). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+)
+
+
+
+@>nul findstr /m "sd_install_complete" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" NEQ "0" (
     @echo sd_weights_downloaded >> ..\scripts\install_status.txt
     @echo sd_install_complete >> ..\scripts\install_status.txt
@@ -312,12 +376,16 @@ echo. > "..\models\stable-diffusion\Put your custom ckpt files here.txt"
 @cd ..\..\..
 @echo PYTHONPATH=%PYTHONPATH%
 
+call where python
+call python --version
+
 @cd ..
 @set SD_UI_PATH=%cd%\ui
 @cd stable-diffusion
 
-@call python --version
+@if NOT DEFINED SD_UI_BIND_PORT set SD_UI_BIND_PORT=9000
+@if NOT DEFINED SD_UI_BIND_IP set SD_UI_BIND_IP=0.0.0.0
+@uvicorn server:app --app-dir "%SD_UI_PATH%" --port %SD_UI_BIND_PORT% --host %SD_UI_BIND_IP%
 
-@uvicorn server:app --app-dir "%SD_UI_PATH%" --port 9000 --host 0.0.0.0
 
-@pause
+@pause

scripts/on_sd_start.sh

@@ -1,15 +1,22 @@
 #!/bin/bash
 
+source ./scripts/functions.sh
+
 cp sd-ui-files/scripts/on_env_start.sh scripts/
+cp sd-ui-files/scripts/bootstrap.sh scripts/
 
-source installer/etc/profile.d/conda.sh
+# activate the installer env
+CONDA_BASEPATH=$(conda info --base)
+source "$CONDA_BASEPATH/etc/profile.d/conda.sh" # avoids the 'shell not initialized' error
 
-cp sd-ui-files/scripts/developer_console.sh .
+conda activate || fail "Failed to activate conda"
+
+# remove the old version of the dev console script, if it's still present
 if [ -e "open_dev_console.sh" ]; then
     rm "open_dev_console.sh"
 fi
 
-python -c "import os; import shutil; frm = 'sd-ui-files/ui/hotfix/9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'; dst = os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'transformers', '9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'); shutil.copyfile(frm, dst) if os.path.exists(dst) else print(''); print('Hotfixed broken JSON file from OpenAI');"
+python -c "import os; import shutil; frm = 'sd-ui-files/ui/hotfix/9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'; dst = os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'transformers', '9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'); shutil.copyfile(frm, dst) if os.path.exists(dst) else print(''); print('Hotfixed broken JSON file from OpenAI');" 
 
 # Caution, this file will make your eyes and brain bleed. It's such an unholy mess.
 # Note to self: Please rewrite this in Python. For the sake of your own sanity.
@@ -21,10 +28,10 @@ if [ -e "scripts/install_status.txt" ] && [ `grep -c sd_git_cloned scripts/insta
 
     git reset --hard
     git pull
-    git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+    git -c advice.detachedHead=false checkout f6cfebffa752ee11a7b07497b8529d5971de916c
 
-    git apply ../ui/sd_internal/ddim_callback.patch
-    git apply ../ui/sd_internal/env_yaml.patch
+    git apply ../ui/sd_internal/ddim_callback.patch || fail "ddim patch failed"
+    git apply ../ui/sd_internal/env_yaml.patch || fail "yaml patch failed"
 
     cd ..
 else
@@ -33,16 +40,14 @@ else
     if git clone https://github.com/basujindal/stable-diffusion.git ; then
         echo sd_git_cloned >> scripts/install_status.txt
     else
-        printf "\n\nError downloading Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "git clone of basujindal/stable-diffusion.git failed"
     fi
 
     cd stable-diffusion
-    git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+    git -c advice.detachedHead=false checkout f6cfebffa752ee11a7b07497b8529d5971de916c
 
-    git apply ../ui/sd_internal/ddim_callback.patch
-    git apply ../ui/sd_internal/env_yaml.patch
+    git apply ../ui/sd_internal/ddim_callback.patch || fail "ddim patch failed"
+    git apply ../ui/sd_internal/env_yaml.patch || fail "yaml patch failed"
 
     cd ..
 fi
@@ -52,37 +57,32 @@ cd stable-diffusion
 if [ `grep -c conda_sd_env_created ../scripts/install_status.txt` -gt "0" ]; then
     echo "Packages necessary for Stable Diffusion were already installed"
 
-    conda activate ./env
+    conda activate ./env || fail "conda activate failed"
 else
     printf "\n\nDownloading packages necessary for Stable Diffusion..\n"
     printf "\n\n***** This will take some time (depending on the speed of the Internet connection) and may appear to be stuck, but please be patient ***** ..\n\n"
 
     # prevent conda from using packages from the user's home directory, to avoid conflicts
     export PYTHONNOUSERSITE=1
+    export PYTHONPATH="$(pwd):$(pwd)/env/lib/site-packages"
 
     if conda env create --prefix env --force -f environment.yaml ; then
         echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "'conda env create' failed"
     fi
 
-    conda activate ./env
+    conda activate ./env || fail "conda activate failed"
 
     if conda install -c conda-forge --prefix ./env -y antlr4-python3-runtime=4.8 ; then
         echo "Installed. Testing.."
     else
-        printf "\n\nError installing antlr4-python3-runtime for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error installing antlr4-python3-runtime"
     fi
 
     out_test=`python -c "import torch; import ldm; import transformers; import numpy; import antlr4; print(42)"`
     if [ "$out_test" != "42" ]; then
-        printf "\n\nDependency test failed! Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Dependency test failed"
     fi
 
     echo conda_sd_env_created >> ../scripts/install_status.txt
@@ -94,20 +94,20 @@ else
     printf "\n\nDownloading packages necessary for GFPGAN (Face Correction)..\n"
 
     export PYTHONNOUSERSITE=1
+    export PYTHONPATH="$(pwd):$(pwd)/env/lib/site-packages"
 
     if pip install -e git+https://github.com/TencentARC/GFPGAN#egg=GFPGAN ; then
         echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error installing the packages necessary for GFPGAN (Face Correction)."
     fi
 
     out_test=`python -c "from gfpgan import GFPGANer; print(42)"`
     if [ "$out_test" != "42" ]; then
-        printf "\n\nDependency test failed! Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        echo "EE The dependency check has failed. This usually means that some system libraries are missing."
+	echo "EE On Debian/Ubuntu systems, this are often these packages: libsm6 libxext6 libxrender-dev"
+	echo "EE Other Linux distributions might have different package names for these libraries."
+        fail "GFPGAN dependency test failed"
     fi
 
     echo conda_sd_gfpgan_deps_installed >> ../scripts/install_status.txt
@@ -119,20 +119,17 @@ else
     printf "\n\nDownloading packages necessary for ESRGAN (Resolution Upscaling)..\n"
 
     export PYTHONNOUSERSITE=1
+    export PYTHONPATH="$(pwd):$(pwd)/env/lib/site-packages"
 
     if pip install -e git+https://github.com/xinntao/Real-ESRGAN#egg=realesrgan ; then
         echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error installing the packages necessary for ESRGAN"
     fi
 
     out_test=`python -c "from basicsr.archs.rrdbnet_arch import RRDBNet; from realesrgan import RealESRGANer; print(42)"`
     if [ "$out_test" != "42" ]; then
-        printf "\n\nDependency test failed! Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "ESRGAN dependency test failed"
     fi
 
     echo conda_sd_esrgan_deps_installed >> ../scripts/install_status.txt
@@ -144,19 +141,16 @@ else
     printf "\n\nDownloading packages necessary for Stable Diffusion UI..\n\n"
 
     export PYTHONNOUSERSITE=1
+    export PYTHONPATH="$(pwd):$(pwd)/env/lib/site-packages"
 
     if conda install -c conda-forge --prefix ./env -y uvicorn fastapi ; then
         echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "'conda install uvicorn' failed" 
     fi
 
     if ! command -v uvicorn &> /dev/null; then
-        printf "\n\nUI packages not found! Error installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "UI packages not found!"
     fi
 
     echo conda_sd_ui_deps_installed >> ../scripts/install_status.txt
@@ -165,7 +159,9 @@ fi
 
 
 mkdir -p "../models/stable-diffusion"
+mkdir -p "../models/vae"
 echo "" > "../models/stable-diffusion/Put your custom ckpt files here.txt"
+echo "" > "../models/vae/Put your VAE files here.txt"
 
 if [ -f "sd-v1-4.ckpt" ]; then
     model_size=`find "sd-v1-4.ckpt" -printf "%s"`
@@ -186,15 +182,10 @@ if [ ! -f "sd-v1-4.ckpt" ]; then
     if [ -f "sd-v1-4.ckpt" ]; then
         model_size=`find "sd-v1-4.ckpt" -printf "%s"`
         if [ ! "$model_size" == "4265380512" ]; then
-            printf "\n\nError: The downloaded model file was invalid! Bytes downloaded: $model_size\n\n"
-            printf "\n\nError downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-            read -p "Press any key to continue"
-            exit
+	    fail "The downloaded model file was invalid! Bytes downloaded: $model_size"
         fi
     else
-        printf "\n\nError downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error downloading the data files (weights) for Stable Diffusion"
     fi
 fi
 
@@ -218,15 +209,10 @@ if [ ! -f "GFPGANv1.3.pth" ]; then
     if [ -f "GFPGANv1.3.pth" ]; then
         model_size=`find "GFPGANv1.3.pth" -printf "%s"`
         if [ ! "$model_size" -eq "348632874" ]; then
-            printf "\n\nError: The downloaded GFPGAN model file was invalid! Bytes downloaded: $model_size\n\n"
-            printf "\n\nError downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-            read -p "Press any key to continue"
-            exit
+            fail "The downloaded GFPGAN model file was invalid! Bytes downloaded: $model_size"
         fi
     else
-        printf "\n\nError downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error downloading the data files (weights) for GFPGAN (Face Correction)."
     fi
 fi
 
@@ -250,15 +236,10 @@ if [ ! -f "RealESRGAN_x4plus.pth" ]; then
     if [ -f "RealESRGAN_x4plus.pth" ]; then
         model_size=`find "RealESRGAN_x4plus.pth" -printf "%s"`
         if [ ! "$model_size" -eq "67040989" ]; then
-            printf "\n\nError: The downloaded ESRGAN x4plus model file was invalid! Bytes downloaded: $model_size\n\n"
-            printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-            read -p "Press any key to continue"
-            exit
+            fail "The downloaded ESRGAN x4plus model file was invalid! Bytes downloaded: $model_size"
         fi
     else
-        printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
-        read -p "Press any key to continue"
-        exit
+        fail "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus"
     fi
 fi
 
@@ -282,13 +263,40 @@ if [ ! -f "RealESRGAN_x4plus_anime_6B.pth" ]; then
     if [ -f "RealESRGAN_x4plus_anime_6B.pth" ]; then
         model_size=`find "RealESRGAN_x4plus_anime_6B.pth" -printf "%s"`
         if [ ! "$model_size" -eq "17938799" ]; then
-            printf "\n\nError: The downloaded ESRGAN x4plus_anime model file was invalid! Bytes downloaded: $model_size\n\n"
-            printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+            fail "The downloaded ESRGAN x4plus_anime model file was invalid! Bytes downloaded: $model_size"
+        fi
+    else
+        fail "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime."
+    fi
+fi
+
+
+if [ -f "../models/vae/vae-ft-mse-840000-ema-pruned.ckpt" ]; then
+    model_size=`find ../models/vae/vae-ft-mse-840000-ema-pruned.ckpt -printf "%s"`
+
+    if [ "$model_size" -eq "334695179" ]; then
+        echo "Data files (weights) necessary for the default VAE (sd-vae-ft-mse-original) were already downloaded"
+    else
+        printf "\n\nThe model file present at models/vae/vae-ft-mse-840000-ema-pruned.ckpt is invalid. It is only $model_size bytes in size. Re-downloading.."
+        rm ../models/vae/vae-ft-mse-840000-ema-pruned.ckpt
+    fi
+fi
+
+if [ ! -f "../models/vae/vae-ft-mse-840000-ema-pruned.ckpt" ]; then
+    echo "Downloading data files (weights) for the default VAE (sd-vae-ft-mse-original).."
+
+    curl -L -k https://huggingface.co/stabilityai/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt > ../models/vae/vae-ft-mse-840000-ema-pruned.ckpt
+
+    if [ -f "../models/vae/vae-ft-mse-840000-ema-pruned.ckpt" ]; then
+        model_size=`find ../models/vae/vae-ft-mse-840000-ema-pruned.ckpt -printf "%s"`
+        if [ ! "$model_size" -eq "334695179" ]; then
+            printf "\n\nError: The downloaded default VAE (sd-vae-ft-mse-original) file was invalid! Bytes downloaded: $model_size\n\n"
+            printf "\n\nError downloading the data files (weights) for the default VAE (sd-vae-ft-mse-original). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
             read -p "Press any key to continue"
             exit
         fi
     else
-        printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        printf "\n\nError downloading the data files (weights) for the default VAE (sd-vae-ft-mse-original). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
@@ -303,15 +311,16 @@ fi
 printf "\n\nStable Diffusion is ready!\n\n"
 
 SD_PATH=`pwd`
-export PYTHONPATH="$SD_PATH;$SD_PATH/env/lib/python3.8/site-packages"
+export PYTHONPATH="$SD_PATH:$SD_PATH/env/lib/python3.8/site-packages"
 echo "PYTHONPATH=$PYTHONPATH"
 
+which python
+python --version
+
 cd ..
 export SD_UI_PATH=`pwd`/ui
 cd stable-diffusion
 
-python --version
-
-uvicorn server:app --app-dir "$SD_UI_PATH" --port 9000 --host 0.0.0.0
+uvicorn server:app --app-dir "$SD_UI_PATH" --port ${SD_UI_BIND_PORT:-9000} --host ${SD_UI_BIND_IP:-0.0.0.0}
 
 read -p "Press any key to continue"

scripts/start.sh

@@ -1,10 +1,22 @@
 #!/bin/bash
 
-source installer/bin/activate
+cd "$(dirname "${BASH_SOURCE[0]}")"
 
-conda-unpack
+# set legacy installer's PATH, if it exists
+if [ -e "installer" ]; then export PATH="$(pwd)/installer/bin:$PATH"; fi
 
-conda --version
-git --version
+# Setup the packages required for the installer
+scripts/bootstrap.sh || exit 1
 
+# set new installer's PATH, if it downloaded any packages
+if [ -e "installer_files/env" ]; then export PATH="$(pwd)/installer_files/env/bin:$PATH"; fi
+
+# Test the bootstrap
+which git
+git --version || exit 1
+
+which conda
+conda --version || exit 1
+
+# Download the rest of the installer and UI
 scripts/on_env_start.sh

ui/index.html

@@ -1,73 +1,79 @@
 <!DOCTYPE html>
 <html>
 <head>
+    <title>Stable Diffusion UI</title>
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <link rel="icon" type="image/png" href="/media/favicon-16x16.png" sizes="16x16">
-    <link rel="icon" type="image/png" href="/media/favicon-32x32.png" sizes="32x32">
-    <link rel="stylesheet" href="/media/main.css?v=21">
-    <link rel="stylesheet" href="/media/modifier-thumbnails.css?v=1">
-    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.2.0/css/all.min.css">
-    <link rel="stylesheet" href="/media/drawingboard.min.css">
-    <script src="/media/jquery-3.6.1.min.js"></script>
-    <script src="/media/drawingboard.min.js"></script>
+    <link rel="icon" type="image/png" href="/media/images/favicon-16x16.png" sizes="16x16">
+    <link rel="icon" type="image/png" href="/media/images/favicon-32x32.png" sizes="32x32">
+    <link rel="stylesheet" href="/media/css/fonts.css?v=1">
+    <link rel="stylesheet" href="/media/css/themes.css?v=3">
+    <link rel="stylesheet" href="/media/css/main.css?v=17">
+    <link rel="stylesheet" href="/media/css/auto-save.css?v=5">
+    <link rel="stylesheet" href="/media/css/modifier-thumbnails.css?v=4">
+    <link rel="stylesheet" href="/media/css/fontawesome-all.min.css?v=1">
+    <link rel="stylesheet" href="/media/css/drawingboard.min.css">
+    <script src="/media/js/jquery-3.6.1.min.js"></script>
+    <script src="/media/js/drawingboard.min.js"></script>
 </head>
 <body>
 <div id="container">
     <div id="top-nav">
         <div id="logo">
-            <h1>Stable Diffusion UI <small>v2.2 <span id="updateBranchLabel"></span></small></h1>
+            <h1>Stable Diffusion UI <small>v2.4.6 <span id="updateBranchLabel"></span></small></h1>
         </div>
-        <ul id="top-nav-items">
-            <li class="dropdown">
-                <span><i class="fa fa-comments icon"></i> Help & Community</span>
-                <ul id="community-links" class="dropdown-content">
-                    <li><a href="https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" target="_blank"><i class="fa-solid fa-circle-question fa-fw"></i> Usual problems and solutions</a></li>
-                    <li><a href="https://discord.com/invite/u9yhsFmEkB" target="_blank"><i class="fa-brands fa-discord fa-fw"></i> Discord user community</a></li>
-                    <li><a href="https://www.reddit.com/r/StableDiffusionUI/" target="_blank"><i class="fa-brands fa-reddit fa-fw"></i> Reddit community</a></li>
-                    <li><a href="https://github.com/cmdr2/stable-diffusion-ui" target="_blank"><i class="fa-brands fa-github fa-fw"></i> Source code on GitHub</a></li>
-                </ul>
-            </li>
-            <li class="dropdown">
+        <div id="server-status">
+            <div id="server-status-color">●</div>
+            <span id="server-status-msg">Stable Diffusion is starting..</span>
+        </div>
+        <div id="tab-container">
+            <span id="tab-main" class="tab active">
+                <span><i class="fa fa-image icon"></i> Generate</span>
+            </span>
+            <span id="tab-settings" class="tab">
                 <span><i class="fa fa-gear icon"></i> Settings</span>
-                <div id="system-settings" class="panel-box settings-box dropdown-content">
-                    <ul id="system-settings-entries">
-                        <li><b class="settings-subheader">System Settings</b></li>
-                        <br/>
-                        <li><input id="save_to_disk" name="save_to_disk" type="checkbox"> <label for="save_to_disk">Automatically save to <input id="diskPath" name="diskPath" size="40" disabled></label></li>
-                        <li><input id="sound_toggle" name="sound_toggle" type="checkbox" checked> <label for="sound_toggle">Play sound on task completion</label></li>
-                        <li><input id="turbo" name="turbo" type="checkbox" checked> <label for="turbo">Turbo mode <small>(generates images faster, but uses an additional 1 GB of GPU memory)</small></label></li>
-                        <li><input id="use_cpu" name="use_cpu" type="checkbox"> <label for="use_cpu">Use CPU instead of GPU <small>(warning: this will be *very* slow)</small></label></li>
-                        <li><input id="use_full_precision" name="use_full_precision" type="checkbox"> <label for="use_full_precision">Use full precision <small>(for GPU-only. warning: this will consume more VRAM)</small></label></li>
-                        <!-- <li><input id="allow_nsfw" name="allow_nsfw" type="checkbox"> <label for="allow_nsfw">Allow NSFW Content (You confirm you are above 18 years of age)</label></li> -->
-                        <br/>
-                        <li><input id="use_beta_channel" name="use_beta_channel" type="checkbox"> <label for="use_beta_channel">🔥Beta channel. Get the latest features immediately (but could be less stable). Please restart the program after changing this.</label></li>
-                    </ul>
-                </div>
-            </li>
-        </ul>
+            </span>
+            <span id="tab-about" class="tab">
+                <span><i class="fa fa-comments icon"></i> Help & Community</span>
+            </span>
+        </div>
     </div>
 
-    <div class="flex-container">
-        <div id="editor" class="col-fixed-10">
-            <div id="server-status">
-                <div id="server-status-color">●</div>
-                <span id="server-status-msg">Stable Diffusion is starting..</span>
-            </div>
+    <div id="tab-content-wrapper">
+        <div id="tab-content-main" class="tab-content active flex-container">
+        <div id="editor">
             <div id="editor-inputs">
                 <div id="editor-inputs-prompt" class="row">
-                    <label for="prompt">Prompt</label>
+                    <label for="prompt"><b>Enter Prompt</b></label> <small>or</small> <button id="promptsFromFileBtn">Load from a file</button>
                     <textarea id="prompt" class="col-free">a photograph of an astronaut riding a horse</textarea>
+                    <input id="prompt_from_file" name="prompt_from_file" type="file" /> <!-- hidden -->
+
+                    <label for="negative_prompt" class="collapsible" id="negative_prompt_handle">
+                        Negative Prompt
+                        <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Writing-prompts#negative-prompts" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip right">Click to learn more about Negative Prompts</span></i></a>
+                        <small>(optional)</small>
+                    </label>
+                    <div class="collapsible-content">
+                        <input id="negative_prompt" name="negative_prompt" placeholder="list the things to remove from the image (e.g. fog, green)">
+                    </div>
                 </div>
 
                 <div id="editor-inputs-init-image" class="row">
-                    <label for="init_image"><b>Initial Image:</b> (optional) </label> <input id="init_image" name="init_image" type="file" /><br/>
+                    <label for="init_image">Initial Image (img2img) <small>(optional)</small> </label> <input id="init_image" name="init_image" type="file" /><br/>
 
                     <div id="init_image_preview_container" class="image_preview_container">
-                        <img id="init_image_preview" src="" width="100" height="100" />
-                        <button class="init_image_clear image_clear_btn">X</button>
+                        <div id="init_image_wrapper">
+                            <img id="init_image_preview" src="" />
+                            <span id="init_image_size_box"></span>
+                            <button class="init_image_clear image_clear_btn">X</button>
+                        </div>
 
                         <br/>
-                        <input id="enable_mask" name="enable_mask" type="checkbox"> <label for="enable_mask">In-Painting (beta) <small>(select the area which the AI will paint into)</small></label>
+                        <input id="enable_mask" name="enable_mask" type="checkbox">
+                        <label for="enable_mask">
+                            In-Painting (beta)
+                            <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Inpainting" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip right">Click to learn more about InPainting</span></i></a>
+                            <small>(select the area which the AI will paint into)</small>
+                        </label>
                         <div id="inpaintingEditor"></div>
                     </div>
                 </div>
@@ -81,32 +87,48 @@
                 <button id="stopImage" class="secondaryButton">Stop All</button>
             </div>
 
-            <div class="line-separator">&nbsp;</div>
+            <span class="line-separator"></span>
 
-            <div id="editor-settings" class="panel-box settings-box">
-                <h4 class="collapsible">Image Settings</h4>
-                <ul id="editor-settings-entries" class="collapsible-content">
-                    <li><b class="settings-subheader">Image Settings</b></li>
-                    <li class="pl-5"><label for="seed">Seed:</label> <input id="seed" name="seed" size="10" value="30000"> <input id="random_seed" name="random_seed" type="checkbox" checked> <label for="random_seed">Random Image</label></li>
-                    <li class="pl-5"><label for="num_outputs_total">Number of images to make:</label> <input id="num_outputs_total" name="num_outputs_total" value="1" size="1"> <label for="num_outputs_parallel">Generate in parallel:</label> <input id="num_outputs_parallel" name="num_outputs_parallel" value="1" size="1"> (images at once)</li>
-                    <li class="pl-5"><label for="stable_diffusion_model">Model:</label>
+            <div id="editor-settings" class="settings-box panel-box">
+                <h4 class="collapsible">
+                    Image Settings
+                    <i id="reset-image-settings" class="fa-solid fa-arrow-rotate-left section-button">
+                        <span class="simple-tooltip left">
+                            Reset Image Settings
+                        </span>
+                    </i>
+                </h4>
+                <div id="editor-settings-entries" class="collapsible-content">
+                    <div><table>
+                    <tr><b class="settings-subheader">Image Settings</b></tr>
+                    <tr class="pl-5"><td><label for="seed">Seed:</label></td><td><input id="seed" name="seed" size="10" value="30000" onkeypress="preventNonNumericalInput(event)"> <input id="random_seed" name="random_seed" type="checkbox" checked><label for="random_seed">Random</label></td></tr>
+                    <tr class="pl-5"><td><label for="num_outputs_total">Number of Images:</label></td><td><input id="num_outputs_total" name="num_outputs_total" value="1" size="1" onkeypress="preventNonNumericalInput(event)"> <label><small>(total)</small></label> <input id="num_outputs_parallel" name="num_outputs_parallel" value="1" size="1" onkeypress="preventNonNumericalInput(event)"> <label for="num_outputs_parallel"><small>(in parallel)</small></label></td></tr>
+                    <tr class="pl-5"><td><label for="stable_diffusion_model">Model:</label></td><td>
                         <select id="stable_diffusion_model" name="stable_diffusion_model">
                             <!-- <option value="sd-v1-4" selected>sd-v1-4</option> -->
                         </select>
-                    </li>
-                    <li id="samplerSelection" class="pl-5"><label for="sampler">Sampler:</label>
+                        <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Custom-Models" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip right">Click to learn more about custom models</span></i></a>
+                    </td></tr>
+                    <tr class="pl-5"><td><label for="vae_model">Custom VAE:</i></label></td><td>
+                        <select id="vae_model" name="vae_model">
+                            <!-- <option value="" selected>None</option> -->
+                        </select>
+                        <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/VAE-Variational-Auto-Encoder" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip right">Click to learn more about VAEs</span></i></a>
+                    </td></tr>
+                    <tr id="samplerSelection" class="pl-5"><td><label for="sampler">Sampler:</label></td><td>
                         <select id="sampler" name="sampler">
-                            <option value="plms" selected>plms</option>
+                            <option value="plms">plms</option>
                             <option value="ddim">ddim</option>
                             <option value="heun">heun</option>
                             <option value="euler">euler</option>
-                            <option value="euler_a">euler_a</option>
+                            <option value="euler_a" selected>euler_a</option>
                             <option value="dpm2">dpm2</option>
                             <option value="dpm2_a">dpm2_a</option>
                             <option value="lms">lms</option>
                         </select>
-                    </li>
-                    <li class="pl-5"><label>Image Size: </label>
+                        <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/How-to-Use#samplers" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip right">Click to learn more about samplers</span></i></a>
+                    </td></tr>
+                    <tr class="pl-5"><td><label>Image Size: </label></td><td>
                         <select id="width" name="width" value="512">
                             <option value="128">128 (*)</option>
                             <option value="192">192</option>
@@ -151,69 +173,150 @@
                             <option value="2048">2048</option>
                         </select>
                         <label for="height"><small>(height)</small></label>
-                    </li>
-                    <li class="pl-5"><label for="num_inference_steps">Number of inference steps:</label> <input id="num_inference_steps" name="num_inference_steps" size="4" value="50"></li>
-                    <li class="pl-5"><label for="guidance_scale_slider">Guidance Scale:</label> <input id="guidance_scale_slider" name="guidance_scale_slider" class="editor-slider" value="75" type="range" min="10" max="500"> <input id="guidance_scale" name="guidance_scale" size="4"></li>
-                    <li class="pl-5"><span id="prompt_strength_container"><label for="prompt_strength_slider">Prompt Strength:</label> <input id="prompt_strength_slider" name="prompt_strength_slider" class="editor-slider" value="80" type="range" min="0" max="99"> <input id="prompt_strength" name="prompt_strength" size="4"><br/></span></li>
-                    <li class="pl-5"><label for="output_format">Output format:</label>
+                    </td></tr>
+                    <tr class="pl-5"><td><label for="num_inference_steps">Inference Steps:</label></td><td> <input id="num_inference_steps" name="num_inference_steps" size="4" value="25" onkeypress="preventNonNumericalInput(event)"></td></tr>
+                    <tr class="pl-5"><td><label for="guidance_scale_slider">Guidance Scale:</label></td><td> <input id="guidance_scale_slider" name="guidance_scale_slider" class="editor-slider" value="75" type="range" min="10" max="500"> <input id="guidance_scale" name="guidance_scale" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"></td></tr>
+                    <tr id="prompt_strength_container" class="pl-5"><td><label for="prompt_strength_slider">Prompt Strength:</label></td><td> <input id="prompt_strength_slider" name="prompt_strength_slider" class="editor-slider" value="80" type="range" min="0" max="99"> <input id="prompt_strength" name="prompt_strength" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"><br/></td></tr></span>
+                    <tr class="pl-5"><td><label for="output_format">Output Format:</label></td><td>
                         <select id="output_format" name="output_format">
                             <option value="jpeg" selected>jpeg</option>
                             <option value="png">png</option>
                         </select>
-                    </li>
-
-                    <br/>
-
-                    <li><b class="settings-subheader">Prompt Settings</b></li>
-                    <li class="pl-5"><label for="negative_prompt">Negative Prompt:</label> <input id="negative_prompt" name="negative_prompt" size="55"></li>
-
-                    <br/>
+                    </td></tr>
+                    </table></div>
 
+                    <div><ul>
                     <li><b class="settings-subheader">Render Settings</b></li>
-                    <li class="pl-5"><input id="stream_image_progress" name="stream_image_progress" type="checkbox"> <label for="stream_image_progress">Show a live preview of the image <small>(uses more VRAM, slightly slower image creation)</small></label></li>
-                    <li class="pl-5"><input id="use_face_correction" name="use_face_correction" type="checkbox" checked> <label for="use_face_correction">Fix incorrect faces and eyes <small>(uses GFPGAN)</small></label></li>
+                    <li class="pl-5"><input id="stream_image_progress" name="stream_image_progress" type="checkbox"> <label for="stream_image_progress">Show a live preview <small>(uses more VRAM, and slower image creation)</small></label></li>
+                    <li class="pl-5"><input id="use_face_correction" name="use_face_correction" type="checkbox"> <label for="use_face_correction">Fix incorrect faces and eyes <small>(uses GFPGAN)</small></label></li>
                     <li class="pl-5">
-                        <input id="use_upscale" name="use_upscale" type="checkbox"> <label for="use_upscale">Upscale the image to 4x resolution using </label>
+                        <input id="use_upscale" name="use_upscale" type="checkbox"> <label for="use_upscale">Upscale image by 4x with </label>
                         <select id="upscale_model" name="upscale_model">
                             <option value="RealESRGAN_x4plus" selected>RealESRGAN_x4plus</option>
                             <option value="RealESRGAN_x4plus_anime_6B">RealESRGAN_x4plus_anime_6B</option>
                         </select>
                     </li>
                     <li class="pl-5"><input id="show_only_filtered_image" name="show_only_filtered_image" type="checkbox" checked> <label for="show_only_filtered_image">Show only the corrected/upscaled image</label></li>
-                    <br/>
-                    <li><small>The system-related settings have been moved to the top-right corner.</small></li>
-                </ul>
+                    </ul></div>
+                </div>
             </div>
 
             <div id="editor-modifiers" class="panel-box">
-                <h4 class="collapsible">Image Modifiers (art styles, tags etc)</h4>
+                <h4 class="collapsible">
+                    Image Modifiers (art styles, tags etc)
+                    <i id="modifier-settings-btn" class="fa-solid fa-gear section-button">
+                        <span class="simple-tooltip left">
+                            Add Custom Modifiers
+                        </span>
+                    </i>
+                </h4>
                 <div id="editor-modifiers-entries" class="collapsible-content">
-                    <label for="preview-image">Image Style:</label>
-                    <select id="preview-image" name="preview-image" value="portrait">
-                        <option value="portrait" selected="">Face</option>
-                        <option value="landscape">Landscape</option>
-                    </select>
-                    &nbsp;
-                    <label for="modifier-card-size-slider">Thumbnail Size:</label>
-                    <input id="modifier-card-size-slider" name="modifier-card-size-slider" value="0" type="range" min="-3" max="5">
+                    <div id="editor-modifiers-entries-toolbar">
+                        <label for="preview-image">Image Style:</label>
+                        <select id="preview-image" name="preview-image" value="portrait">
+                            <option value="portrait" selected="">Face</option>
+                            <option value="landscape">Landscape</option>
+                        </select>
+                        &nbsp;
+                        <label for="modifier-card-size-slider">Thumbnail Size:</label>
+                        <input id="modifier-card-size-slider" name="modifier-card-size-slider" value="0" type="range" min="-3" max="5">
+                    </div>
                 </div>
             </div>
         </div>
 
         <div id="preview" class="col-free">
             <div id="initial-text">
-                Type a prompt and press the "Make Image" button.<br/><br/>You can set an "Initial Image" if you want to guide the AI.<br/><br/>You can also add modifiers like "Realistic", "Pencil Sketch", "ArtStation" etc by browsing through the "Image Modifiers" section and selecting the desired modifiers.<br/><br/>Click "Advanced Settings" for additional settings like seed, image size, number of images to generate etc.<br/><br/>Enjoy! :)
+                Type a prompt and press the "Make Image" button.<br/><br/>You can set an "Initial Image" if you want to guide the AI.<br/><br/>
+                You can also add modifiers like "Realistic", "Pencil Sketch", "ArtStation" etc by browsing through the "Image Modifiers" section 
+                and selecting the desired modifiers.<br/><br/>
+                Click "Image Settings" for additional settings like seed, image size, number of images to generate etc.<br/><br/>Enjoy! :)
             </div>
             <div id="preview-tools">
                 <button id="clear-all-previews" class="secondaryButton"><i class="fa-solid fa-trash-can"></i> Clear All</button>
             </div>
         </div>
+        </div>
+
+        <div id="tab-content-settings" class="tab-content">
+            <div id="system-settings" class="tab-content-inner">
+                <h1>System Settings</h1>
+                <table class="form-table"></table>
+                <br/>
+                <button id="save-system-settings-btn" class="primaryButton">Save</button>
+                <br/><br/>
+                <div>
+                    <h3><i class="fa fa-microchip icon"></i> System Info</h3>
+                    <div id="system-info"></div>
+                </div>
+            </div>
+        </div>
+        <div id="tab-content-about" class="tab-content">
+            <div class="tab-content-inner">
+                <div class="float-container">
+                    <div class="float-child">
+                        <h1>Help</h1>
+                        <ul id="help-links">
+                            <li><span class="help-section">Using the software</span>
+                            <ul>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/How-To-Use" target="_blank"><i class="fa-solid fa-book fa-fw"></i> How to use</a>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/UI-Overview" target="_blank"><i class="fa-solid fa-list fa-fw"></i> UI Overview</a>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Writing-Prompts" target="_blank"><i class="fa-solid fa-pen-to-square fa-fw"></i> Writing prompts</a>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Inpainting" target="_blank"><i class="fa-solid fa-paintbrush fa-fw"></i> Inpainting</a>
+                            </ul>
+  
+                            <li><span class="help-section">Installation</span>
+                            <ul>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting" target="_blank"><i class="fa-solid fa-circle-question fa-fw"></i> Troubleshooting</a>
+                            </ul>
+  
+                            <li><span class="help-section">Downloadable Content</span>
+                            <ul>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/Custom-Models" target="_blank"><i class="fa-solid fa-images fa-fw"></i> Custom Models</a>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/UI-Plugins" target="_blank"><i class="fa-solid fa-puzzle-piece fa-fw"></i> UI Plugins</a>
+                                <li> <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/VAE-Variational-Auto-Encoder" target="_blank"><i class="fa-solid fa-hand-sparkles fa-fw"></i> VAE Variational Auto Encoder</a>
+                            </ul>
+                        </ul>
+                    </div>
+  
+                    <div class="float-child">
+                        <h1>Community</h1>
+                        <ul id="community-links">
+                            <li><a href="https://discord.com/invite/u9yhsFmEkB" target="_blank"><i class="fa-brands fa-discord fa-fw"></i> Discord user community</a></li>
+                            <li><a href="https://www.reddit.com/r/StableDiffusionUI/" target="_blank"><i class="fa-brands fa-reddit fa-fw"></i> Reddit community</a></li>
+                            <li><a href="https://github.com/cmdr2/stable-diffusion-ui" target="_blank"><i class="fa-brands fa-github fa-fw"></i> Source code on GitHub</a></li>
+                        </ul>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+    
+
+    <div id="save-settings-config" class="popup">
+        <div>
+            <i class="close-button fa-solid fa-xmark"></i>
+            <h1>Save Settings Configuration</h1>
+            <p>Select which settings should be remembered when restarting the browser</p>
+            <table id="save-settings-config-table" class="form-table">
+            </table>
+        </div>
     </div>
 
-    <div class="line-separator">&nbsp;</div>
+    <div id="modifier-settings-config" class="popup">
+        <div>
+            <i class="close-button fa-solid fa-xmark"></i>
+            <h1>Modifier Settings</h1>
+            <p>Set your custom modifiers (one per line)</p>
+            <textarea id="custom-modifiers-input" placeholder="Enter your custom modifiers, one-per-line"></textarea>
+            <p><small><b>Tip:</b> You can include special characters like {} () [] and |. You can also put multiple comma-separated phrases in a single line, to make a single modifier that combines all of those.</small></p>
+        </div>
+    </div>
 
-    <div id="footer" class="panel-box">
-        <p>If you found this project useful and want to help keep it alive, please <a href="https://ko-fi.com/cmdr2_stablediffusion_ui" target="_blank"><img src="media/kofi.png" id="coffeeButton"></a> to help cover the cost of development and maintenance! Thank you for your support!</p>
+    <div id="footer-spacer"></div>
+    <div id="footer">
+        <div class="line-separator">&nbsp;</div>
+        <p>If you found this project useful and want to help keep it alive, please <a href="https://ko-fi.com/cmdr2_stablediffusion_ui" target="_blank"><img src="/media/images/kofi.png" id="coffeeButton"></a> to help cover the cost of development and maintenance! Thank you for your support!</p>
         <p>Please feel free to join the <a href="https://discord.com/invite/u9yhsFmEkB" target="_blank">discord community</a> or <a href="https://github.com/cmdr2/stable-diffusion-ui/issues" target="_blank">file an issue</a> if you have any problems or suggestions in using this interface.</p>
         <div id="footer-legal">
             <p><b>Disclaimer:</b> The authors of this project are not responsible for any content generated using this interface.</p>
@@ -224,13 +327,24 @@
 </div>
 </body>
 
-<script src="media/main.js?v=31"></script>
+<script src="media/js/parameters.js?v=9"></script>
+<script src="media/js/plugins.js?v=1"></script>
+<script src="media/js/utils.js?v=6"></script>
+<script src="media/js/inpainting-editor.js?v=1"></script>
+<script src="media/js/image-modifiers.js?v=6"></script>
+<script src="media/js/auto-save.js?v=8"></script>
+<script src="media/js/main.js?v=22.1"></script>
+<script src="media/js/themes.js?v=4"></script>
+<script src="media/js/dnd.js?v=9"></script>
 <script>
 async function init() {
-    await loadModifiers()
+    await initSettings()
+    await getModels()
     await getDiskPath()
     await getAppConfig()
-    await getModels()
+    await loadModifiers()
+    await loadUIPlugins()
+    await getDevices()
 
     setInterval(healthCheck, HEALTH_PING_INTERVAL * 1000)
     healthCheck()

ui/media/css/auto-save.css

@@ -0,0 +1,48 @@
+/* Auto-Settings Styling */
+#auto_save_settings ~ button {
+    margin: 5px;
+}
+#auto_save_settings:not(:checked) ~ button {
+    display: none;
+}
+
+.form-table {
+    margin: auto;
+}
+
+.form-table th {
+    padding-top: 15px;
+    padding-bottom: 5px;
+}
+
+.form-table td:first-child > *,
+.form-table th:first-child > * {
+    float: right;
+    white-space: nowrap;
+}
+
+.form-table td:last-child > *,
+.form-table th:last-child > * {
+    float: left;
+}
+
+.form-table small {
+    color: rgb(153, 153, 153);
+}
+
+#system-settings .form-table td {
+    height: 24px;
+}
+
+#system-settings .form-table td:last-child div {
+    display: flex;
+    align-items: center;
+}
+#system-settings .form-table td:last-child div > :not([type="checkbox"]):first-child {
+    margin-left: 3px;
+}
+
+#system-settings .form-table td:last-child div small {
+    padding-left: 5px;
+    text-align: left;
+}

ui/media/css/fonts.css

@@ -0,0 +1,40 @@
+/* work-sans-regular - latin */
+@font-face {
+    font-family: 'Work Sans';
+    font-style: normal;
+    font-weight: 400;
+    src: local(''),
+         url('/media/fonts/work-sans-v18-latin-regular.woff2') format('woff2'), /* Chrome 26+, Opera 23+, Firefox 39+ */
+         url('/media/fonts/work-sans-v18-latin-regular.woff') format('woff'); /* Chrome 6+, Firefox 3.6+, IE 9+, Safari 5.1+ */
+  }
+  
+  /* work-sans-600 - latin */
+  @font-face {
+    font-family: 'Work Sans';
+    font-style: normal;
+    font-weight: 600;
+    src: local(''),
+         url('/media/fonts/work-sans-v18-latin-600.woff2') format('woff2'), /* Chrome 26+, Opera 23+, Firefox 39+ */
+         url('/media/fonts/work-sans-v18-latin-600.woff') format('woff'); /* Chrome 6+, Firefox 3.6+, IE 9+, Safari 5.1+ */
+  }
+  
+  /* work-sans-700 - latin */
+  @font-face {
+    font-family: 'Work Sans';
+    font-style: normal;
+    font-weight: 700;
+    src: local(''),
+         url('/media/fonts/work-sans-v18-latin-700.woff2') format('woff2'), /* Chrome 26+, Opera 23+, Firefox 39+ */
+         url('/media/fonts/work-sans-v18-latin-700.woff') format('woff'); /* Chrome 6+, Firefox 3.6+, IE 9+, Safari 5.1+ */
+  }
+  
+  /* work-sans-800 - latin */
+  @font-face {
+    font-family: 'Work Sans';
+    font-style: normal;
+    font-weight: 800;
+    src: local(''),
+         url('/media/fonts/work-sans-v18-latin-800.woff2') format('woff2'), /* Chrome 26+, Opera 23+, Firefox 39+ */
+         url('/media/fonts/work-sans-v18-latin-800.woff') format('woff'); /* Chrome 6+, Firefox 3.6+, IE 9+, Safari 5.1+ */
+  }
+  

ui/media/css/main.css

@@ -0,0 +1,915 @@
+* {
+    font-family: Work Sans, Verdana, Geneva, sans-serif;
+    box-sizing: border-box;
+}
+
+html {
+    position: relative;
+}
+
+body {
+    margin: 0;
+    font-size: 11pt;
+    background-color: var(--background-color1);
+    color: var(--text-color);
+}
+a {
+    color: rgb(0, 102, 204);
+}
+a:visited {
+    color: rgb(0, 102, 204);
+}
+label {
+    font-size: 10pt;
+}
+#prompt {
+    width: 100%;
+    height: 65pt;
+    font-size: 13px;
+    margin-bottom: 6px;
+    margin-top: 5px;
+    display: block;
+}
+.image_preview_container {
+    margin-top: 10pt;
+}
+.image_clear_btn {
+    position: absolute;
+    transform: translate(30%, -30%);
+    background: black;
+    color: white;
+    border: 2pt solid #ccc;
+    padding: 0;
+    cursor: pointer;
+    outline: inherit;
+    border-radius: 8pt;
+    width: 16pt;
+    height: 16pt;
+    font-family: Verdana;
+    font-size: 8pt;
+    top: 0px;
+    right: 0px;
+}
+.settings-box ul {
+    font-size: 9pt;
+    margin-bottom: 5px;
+    padding-left: 10px;
+    list-style-type: none;
+}
+.settings-box li {
+    padding-bottom: 4pt;
+}
+.editor-slider {
+    vertical-align: middle;
+}
+.outputMsg {
+    font-size: small;
+    padding-bottom: 3pt;
+}
+#footer {
+    font-size: small;
+    padding: 10pt;
+    background: none;
+}
+#footer-legal {
+    font-size: 8pt;
+}
+#footer-spacer {
+    flex: 0.7
+}
+.imgSeedLabel {
+    font-size: 0.8em;
+    background-color: var(--background-color2);
+    border-radius: 3px;
+    padding: 5px;
+}
+.imgItem {
+    display: inline-block;
+    margin-top: 1em;
+    margin-right: 1em;
+}
+.imgContainer {
+    display: flex;
+    justify-content: flex-end;
+}
+.imgItemInfo {
+    padding-bottom: 0.5em;
+    display: flex;
+    align-items: flex-end;
+    flex-direction: column;
+    position: absolute;
+    padding: 5px;
+    opacity: 0;
+    transition: 0.1s all;
+}
+.imgContainer:hover > .imgItemInfo {
+    opacity: 1;
+}
+.imgItemInfo * {
+    margin-bottom: 7px;
+}
+#container {
+    min-height: 100vh;
+    width: 100%;
+    margin: 0px;
+    display: flex;
+    flex-direction: column;
+}
+#logo small {
+    font-size: 11pt;
+}
+#editor {
+    background: var(--background-color1);
+    padding: 16px;
+    display: flex;
+    flex-direction: column;
+    flex: 0 0 370pt;
+}
+#editor label {
+    font-weight: normal;
+}
+#editor h4 {
+    margin: 0px;
+    white-space: nowrap;
+}
+#editor .collapsible-content {
+    width: 100%;
+}
+.settings-box label small {
+    color: rgb(153, 153, 153);
+    margin-right: 10px;
+}
+#preview {
+    padding: 8px;
+    background: var(--background-color1);
+}
+#preview .collapsible-content {
+    padding: 0px 15px;
+}
+#editor-inputs-prompt {
+    flex: 1;
+}
+#editor-inputs .row {
+    padding-bottom: 10px;
+}
+#makeImage {
+    border-radius: 6px;
+}
+#editor-modifiers h5 {
+    padding: 5pt 0;
+    margin: 0;
+}
+#makeImage {
+    flex: 0 0 70px;
+    background: var(--accent-color);
+    border: var(--primary-button-border);
+    color: rgb(255, 221, 255);
+    width: 100%;
+    height: 30pt;
+}
+#makeImage:hover {
+    background: hsl(var(--accent-hue), 100%, calc(var(--accent-lightness) + 6%));
+}
+#stopImage {
+    flex: 0 0 70px;
+    background: rgb(132, 8, 0);
+    border: 2px solid rgb(122, 29, 0);
+    color: rgb(255, 221, 255);
+    width: 100%;
+    height: 30pt;
+    border-radius: 6px;
+    display: none;
+    margin-top: 2pt;
+}
+#stopImage:hover {
+    background: rgb(177, 27, 0);
+}
+.flex-container {
+    display: flex;
+    width: 100%;
+}
+.col-free {
+    flex: 1;
+}
+.collapsible {
+    cursor: pointer;
+}
+.collapsible-content {
+    display: block;
+    padding-left: 15px;
+}
+.collapsible-content h5 {
+    padding: 5pt 0pt;
+    margin: 0;
+    font-size: 10pt;
+}
+.collapsible-handle {
+    color: white;
+    padding-right: 5px;
+}
+.collapsible:not(.active) ~ .collapsible-content  {
+    display: none !important;
+}
+#editor-modifiers {
+    max-width: 600px;
+    overflow-y: auto;
+    overflow-x: hidden;
+}
+#editor-modifiers .editor-modifiers-leaf {
+    padding-top: 10pt;
+    padding-bottom: 10pt;
+}
+img {
+    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
+}
+.line-separator {
+    background: var(--background-color3);
+    height: 1pt;
+    margin: 16px 0px;
+}
+#editor-inputs-tags-container {
+    margin-top: 5pt;
+    display: none;
+}
+#server-status {
+    position: absolute;
+    right: 16px;
+    top: 50%;
+    transform: translateY(-50%);
+    text-align: right;
+}
+#server-status-color {
+    font-size: 14pt;
+    color: rgb(200, 139, 0);
+    display: inline;
+}
+#server-status-msg {
+    color: rgb(200, 139, 0);
+    padding-left: 2pt;
+    font-size: 10pt;
+}
+.preview-prompt {
+    font-size: 13pt;
+    margin-bottom: 10pt;
+}
+#coffeeButton {
+    height: 23px;
+    transform: translateY(25%);
+}
+
+#inpaintingEditor {
+    width: 300pt;
+    height: 300pt;
+    margin-top: 5pt;
+}
+.drawing-board-canvas-wrapper {
+    background-size: 100% 100%;
+}
+.drawing-board-controls {
+    min-width: 273px;
+}
+.drawing-board-control > button {
+    background-color: #eee;
+    border-radius: 3pt;
+}
+.drawing-board-control-inner {
+    background-color: #eee;
+    border-radius: 3pt;
+}
+#inpaintingEditor canvas {
+    opacity: 0.6;
+}
+#enable_mask {
+    margin-top: 8pt;
+}
+
+#top-nav {
+    position: relative;
+    background: var(--background-color4);
+    display: flex;
+}
+.tab .icon {
+    padding-right: 4pt;
+    font-size: 14pt;
+    transform: translateY(1pt);
+}
+#logo {
+    display: inline;
+    padding: 12px;
+    white-space: nowrap;
+}
+#logo h1 {
+    display: inline;
+}
+#top-nav-items {
+    list-style-type: none;
+    display: inline;
+    float: right;
+}
+#top-nav-items > li {
+    float: left;
+    display: inline;
+    padding-left: 20pt;
+}
+#top-nav-items > li:first-child {
+    cursor: default;
+}
+#initial-text {
+    padding-top: 15pt;
+    padding-left: 4pt;
+}
+.settings-subheader {
+    font-size: 10pt;
+    font-weight: bold;
+}
+.pl-5 {
+    padding-left: 5pt;
+}
+#community-links {
+    display: inline-block;
+    list-style-type: none;
+    text-align: left;
+    margin: auto;
+    padding: 0px;
+}
+#community-links li {
+    padding-bottom: 12pt;
+    display: block;
+    font-size: 10pt;
+}
+#community-links li .fa-fw {
+    padding-right: 2pt;
+}
+#community-links li a {
+    color: var(--text-color);
+    text-decoration: none;
+}
+.float-child h1 {
+    border-bottom: var(--button-border);
+}
+#help-links {
+    display: inline-block;
+    list-style-type: none;
+    text-align: left;
+    margin: auto;
+    padding: 0px;
+}
+#help-links li {
+    padding-bottom: 12pt;
+    display: block;
+    font-size: 10pt;
+}
+#help-links li .fa-fw {
+    padding-right: 2pt;
+}
+#help-links li a {
+    color: var(--text-color);
+    text-decoration: none;
+}
+#help-links li ul {
+    padding-inline-start: 10px;
+    margin-top: 8px;
+}
+.help-section {
+    font-size: 130%;
+}
+.dropdown {
+    overflow: hidden;
+}
+.dropdown-content {
+    display: none;
+    position: absolute;
+    z-index: 2;
+
+    background: var(--background-color4);
+    border: 2px solid var(--background-color2);
+    border-radius: 7px;
+    padding: 5px;
+    margin-bottom: 15px;
+    box-shadow: 0 20px 28px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
+}
+.dropdown:hover .dropdown-content {
+    display: block;
+}
+
+.imageTaskContainer {
+    border: 1px solid var(--background-color2);
+    margin-bottom: 10pt;
+    padding: 5pt;
+    border-radius: 5pt;
+    box-shadow: 0 20px 28px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
+}
+.imageTaskContainer > div > .collapsible-handle {
+    display: none;
+}
+.taskStatusLabel {
+    float: left;
+    font-size: 8pt;
+    background:var(--background-color2);
+    border: 1px solid rgb(61, 62, 66);
+    padding: 2pt 4pt;
+    border-radius: 2pt;
+    margin-right: 5pt;
+}
+.activeTaskLabel {
+    background:rgb(0, 90, 30);
+    border: 1px solid rgb(0, 75, 19);
+    color:rgb(222, 253, 230)
+}
+.waitingTaskLabel {
+    background:rgb(128, 89, 0);
+    border: 1px solid rgb(107, 75, 0);
+    color:rgb(255, 242, 211)
+}
+.primaryButton {
+    flex: 0 0 70px;
+    background: var(--accent-color);
+    border: var(--primary-button-border);
+    color: rgb(255, 221, 255);
+}
+.secondaryButton {
+    background: rgb(132, 8, 0);
+    border: 1px solid rgb(122, 29, 0);
+    color: rgb(255, 221, 255);
+    padding: 3pt 6pt;
+    border-radius: 5px;
+}
+.secondaryButton:hover {
+    background: rgb(177, 27, 0);
+}
+.stopTask {
+    float: right;
+}
+#preview-tools {
+    display: none;
+    padding: 4pt;
+}
+.taskConfig {
+    font-size: 10pt;
+    color: #aaa;
+    margin-bottom: 5pt;
+}
+.img-batch {
+    display: inline;
+}
+#prompt_from_file {
+    display: none;
+}
+#init_image_preview {
+    max-width: 150px;
+    max-height: 150px;
+    height: 100%;
+    width: 100%;
+	object-fit: contain;
+    border-radius: 6px;
+    transition: all 1s ease-in-out;
+}
+
+#init_image_preview:hover {
+    max-width: 500px;
+    max-height: 1000px;
+
+    transition: all 1s 0.5s ease-in-out;
+}
+
+#init_image_wrapper {
+    position: relative;
+    width: fit-content;
+}
+
+#init_image_size_box {
+    position: absolute;
+    right: 0px;
+    bottom: 3px;
+    padding: 3px;
+    background: black;
+    color: white;
+    text-shadow: 0px 0px 4px black;
+    opacity: 60%;
+    font-size: 12px;
+    border-radius: 6px 0px;
+}
+
+#editor-settings-entries {
+    display: flex;
+    flex-direction: column;
+}
+
+#editor-settings-entries > div {
+    margin-top: 15px;
+}
+
+#editor-settings-entries ul {
+    margin: 0px;
+    padding: 0px;
+}
+
+#editor-settings-entries table td {
+    padding: 0px;
+    line-height: 28px;
+}
+
+#editor-settings-entries table td:first-child {
+    float: right;
+    padding-right: 4px;
+    white-space: nowrap;
+}
+
+#negative_prompt {
+    width: 100%;
+}
+
+/* INPUTS STYLING */
+button,
+input[type="file"],
+input[type="checkbox"],
+select,
+option {
+    cursor: pointer;
+}
+
+input,
+select,
+textarea {
+    border-radius: var(--input-border-radius);
+    padding: 4px;
+    accent-color: var(--accent-color);
+    background: var(--input-background-color);
+    border: var(--input-border-size) solid var(--input-border-color); 
+    color: var(--input-text-color);
+    font-size: 9pt;
+}
+
+input:hover {
+    accent-color: var(--accent-color-hover);
+}
+
+input { 
+    padding: 4px 6px;
+}
+
+input:focus,
+select:focus,
+textarea:focus {
+    outline: 2px solid var(--accent-color);
+}
+
+input[disabled],
+select[disabled],
+textarea[disabled] {
+    opacity: 0.5;
+}
+
+input[type="file"] {
+    width: 100%;
+    padding: 2px;
+}
+
+button,
+input::file-selector-button { 
+    padding: 2px 4px;
+    border-radius: 4px;
+    background: var(--button-color);
+    color: var(--button-text-color);
+    border: var(--button-border);
+}
+
+input::file-selector-button {
+    padding: 0px 4px;
+    height: 19px;
+}
+
+/* MOBILE SUPPORT */
+@media screen and (max-width: 700px) {
+    #top-nav {
+        flex-direction: column;
+    }
+    body {
+        margin: 0px;
+    }
+    #container {
+        margin: 0px;
+    }
+    .flex-container {
+        flex-direction: column;
+    }
+    #preview {
+        margin: 0px;
+        padding: 0px;
+    }
+    #preview .collapsible-content {
+        padding: 0px;
+    }
+    #preview .collapsible-content {
+        padding: 0px;
+    }
+    .imgItem {
+        margin-right: 0px;
+    }
+    .imgItem img {
+        height: 100%;
+        width: 100%;
+        object-fit: contain;
+    }
+    .dropdown-content {
+        width: auto !important;
+        transform: none !important;
+        left: 0px;
+        right: 0px;
+    }
+    #editor {
+        padding: 16px 8px;
+    }
+    .tab-content-inner {
+        margin: 0px;
+    }
+    .tab {
+        font-size: 0;
+    }
+    .tab .icon {
+        padding-right: 0px;
+    }
+    #server-status {
+        display: none;
+    }
+    .popup > div {
+        padding-left: 5px !important;
+        padding-right: 5px !important;
+    }
+    .popup > div input, .popup > div select {
+        max-width: 40vw;
+    }
+    .popup .close-button {
+        padding: 0px !important;
+        margin: 24px !important;
+    }
+    .simple-tooltip.right {
+        right: initial;
+        left: 0px;
+        top: 50%;
+        transform: translate(calc(-100% + 15%), -50%);
+    }
+    :hover > .simple-tooltip.right {
+        transform: translate(100%, -50%);
+    }
+}
+
+@media (min-width: 700px) {
+    /* #editor {
+        max-width: 480px;
+    } */
+    .float-container {
+        padding: 20px;
+    }
+    .float-child {
+        width: 50%;
+        float: left;
+        padding: 20px;
+    }
+}
+
+.help-btn {
+    position: relative;
+}
+
+#promptsFromFileBtn {
+    font-size: 9pt;
+}
+
+.section-button {
+    position: relative;
+    transform: translateY(-13%);
+}
+.collapsible:not(.active) #copy-image-settings {
+    display: none;
+}
+
+.section-button {
+    cursor: pointer;
+    float: right;
+    padding: 8px;
+    opacity: 1;
+    transition: opacity 0.5;
+}
+
+.section-button {
+    cursor: pointer;
+    float: right;
+    padding: 8px;
+    opacity: 1;
+    transition: opacity 0.5;
+}
+
+.collapsible:not(.active) .section-button {
+    display: none;
+}
+
+/* SIMPLE TOOTIP */
+.simple-tooltip {
+	border-radius: 3px;
+	font-weight: bold;
+	font-size: 12px;
+    background-color: var(--background-color3);
+
+	visibility: hidden;
+	opacity: 0;
+	position: absolute;
+	white-space: nowrap;
+	padding: 8px 12px;
+	transition: 0.3s all;
+
+    pointer-events: none;
+}
+
+@media (hover: hover) {
+    :hover > .simple-tooltip {
+        opacity: 1;
+        visibility: visible;
+    }
+}
+.simple-tooltip.right {
+    right: 0px;
+    top: 50%;
+    transform: translate(calc(100% - 15%), -50%);
+}
+:hover > .simple-tooltip.right {
+    transform: translate(100%, -50%);
+}
+
+.simple-tooltip.top {
+    top: 0px;
+    left: 50%;
+    transform: translate(-50%, calc(-100% + 15%));
+}
+:hover > .simple-tooltip.top {
+    transform: translate(-50%, -100%);
+}
+
+.simple-tooltip.left {
+    left: 0px;
+    top: 50%;
+    transform: translate(calc(-100% + 15%), -50%);
+}
+:hover > .simple-tooltip.left {
+    transform: translate(-100%, -50%);
+}
+
+.simple-tooltip.bottom {
+    bottom: 0px;
+    left: 50%;
+    transform: translate(-50%, calc(100% - 15%));
+}
+:hover > .simple-tooltip.bottom {
+    transform: translate(-50%, 100%);
+}
+
+/* PROGRESS BAR */
+.progress-bar {
+    background: var(--background-color3);
+    border-radius: 4px;
+    border: 2px solid var(--background-color3);
+    height: 16px;
+    position: relative;
+    transition: 0.25s 1s border, 0.25s 1s height;
+}
+.progress-bar > div {
+    background: var(--accent-color);
+    border-radius: 4px;
+    position: absolute;
+    left: 0;
+    top: 0;
+    bottom: 0;
+    width: 0%;
+    transition: width 1s ease-in-out;
+}
+.progress-bar.active {
+    background: repeating-linear-gradient(-65deg, 
+        var(--background-color2), 
+        var(--background-color2) 4px, 
+        var(--background-color3) 5px, 
+        var(--background-color3) 9px, 
+        var(--background-color2) 10px);
+    background-size: 200% auto;
+    background-position: 0 100%;
+    animation: progress-anim 2s infinite;
+    animation-fill-mode: forwards;
+    animation-timing-function: linear;
+}
+
+@keyframes progress-anim { 
+    0%   { background-position: -55px 0; }
+    100% { background-position: 0 0; }
+}
+
+/* POPUPS */
+.popup:not(.active) {
+    visibility: hidden;
+    opacity: 0;
+}
+
+.popup {
+    position: absolute;
+    background: rgba(32, 33, 36, 50%);
+    top: 0px;
+    left: 0px;
+    right: 0px;
+    bottom: 0px;
+    z-index: 1000;
+    opacity: 1;
+    transition: 0s visibility, 0.3s opacity;
+}
+
+@media only screen and (min-height: 1050px) {
+    .popup {
+        position: fixed;
+    }
+}
+
+.popup > div {
+    position: relative;
+    background: var(--background-color2);
+    border: solid 1px var(--background-color3);
+    max-width: 700px;
+    margin: auto;
+    margin-top: 50px;
+    border-radius: 6px;
+    padding: 30px;
+    text-align: center;
+    box-shadow: 0px 0px 30px black;
+}
+
+.popup .close-button {
+    position: absolute;
+    right: 0px;
+    top: 0px;
+    transform: scale(150%);
+    cursor: pointer;
+    padding: 24px;
+}
+
+/* TABS */
+#tab-container {
+    display: flex;
+    align-items: flex-end;
+}
+
+.tab {
+    padding: 8px 16px;
+    border-radius: 4px 4px 0px 0px;
+    margin-left: 8px;
+    cursor: pointer;
+    background: var(--background-color1);
+    opacity: 50%;
+    transition: opacity 0.25s;
+}
+
+.tab:hover {
+    opacity: 75%;
+}
+
+.tab.active {
+    opacity: 100%;
+}
+
+.tab-content:not(.active) {
+    display: none;
+}
+
+#tab-content-wrapper {
+    border-top: 8px solid var(--background-color1);
+}
+
+.tab-content-inner {
+    margin: auto;
+    max-width: 600px;
+    text-align: center;
+    padding: 20px 10px;
+}
+
+.panel-box {
+    background: var(--background-color2);
+    border: 1px solid var(--background-color3);
+    border-radius: 7px;
+    padding: 7px;
+    margin-bottom: 15px;
+    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
+}
+
+i.active {
+    background: var(--accent-color);
+}
+#system-info {
+    max-width: 800px;
+    font-size: 10pt;
+}
+#system-info .value {
+    text-align: left;
+    padding-left: 10pt;
+}
+#system-info label {
+    float: right;
+    font-weight: bold;
+}
+#save-system-settings-btn {
+    padding: 4pt 8pt;
+}

ui/media/css/modifier-thumbnails.css

@@ -214,3 +214,10 @@
     margin-bottom: 0.5em;
     vertical-align: middle;
 }
+#modifier-settings-btn {
+    float: right;
+}
+#modifier-settings-config textarea {
+    width: 90%;
+    height: 150px;
+}

ui/media/css/themes.css

@@ -0,0 +1,146 @@
+:root {
+    --background-color1: rgb(32, 33, 36); /* main parts of the page */
+    --background-color2: rgb(44, 45, 48); /* main panels */
+    --background-color3: rgb(47, 49, 53);
+    --background-color4: rgb(18, 18, 19); /* settings dropdowns */
+
+    --accent-hue: 266;
+    --accent-lightness: 36%;
+    --accent-lightness-hover: 40%;
+
+    --text-color: #eee;
+
+    --input-text-color: black;
+    --input-background-color: #e9e9ed; 
+    --input-border-color: #8f8f9d; 
+
+    --button-text-color: var(--input-text-color);
+    --button-color: #e9e9ed;
+    --button-border: 1px solid #8f8f9d;
+
+    /* other */
+    --input-border-radius: 4px;
+    --input-border-size: 1px; 
+    --accent-color: hsl(var(--accent-hue), 100%, var(--accent-lightness));
+    --accent-color-hover: hsl(var(--accent-hue), 100%, var(--accent-lightness-hover));
+    --primary-button-border: none;
+}
+
+.theme-light {
+    --background-color1: white; 
+    --background-color2: #ececec;
+    --background-color3: #e7e9eb;
+    --background-color4: #cccccc;
+
+    --text-color: black;
+
+    --input-text-color: black;
+    --input-background-color: #f8f9fa; 
+    --input-border-color: grey; 
+}
+
+.theme-discord {
+    --background-color1: #36393f;
+    --background-color2: #2f3136;
+    --background-color3: #292b2f;
+    --background-color4: #202225;
+    
+    --accent-hue: 235;
+    --accent-lightness: 65%;
+    --primary-button-border: none;
+    
+    --button-color: var(--accent-color);
+    --button-border: none;
+
+    --input-text-color: #ccc;
+    --input-border-size: 2px; 
+    --input-background-color: #202225; 
+    --input-border-color: var(--input-background-color); 
+}
+
+.theme-cool-blue {
+    --main-hue: 222;
+	--main-saturation: 18%;
+	--value-base: 18%;
+	--value-step: 3%;
+	--background-color1: hsl(var(--main-hue), var(--main-saturation), var(--value-base));
+	--background-color2: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (1 * var(--value-step))));
+	--background-color3: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (2 * var(--value-step))));
+	--background-color4: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (3 * var(--value-step))));
+    
+    --accent-hue: 212;
+    --primary-button-border: none;
+
+    --button-color: var(--accent-color);
+    --button-border: none;
+    
+    --input-border-size: 1px; 
+    --input-background-color: var(--background-color3);
+	--input-text-color: #ccc;
+	--input-border-color: var(--background-color4);
+}
+
+
+.theme-blurple {
+    --main-hue: 235;
+	--main-saturation: 18%;
+	--value-base: 16%;
+	--value-step: 3%;
+	--background-color1: hsl(var(--main-hue), var(--main-saturation), var(--value-base));
+	--background-color2: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (1 * var(--value-step))));
+	--background-color3: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (2 * var(--value-step))));
+	--background-color4: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (3 * var(--value-step))));
+    
+    --primary-button-border: none;
+
+    --button-color: var(--accent-color);
+    --button-border: none;
+    
+    --input-border-size: 1px; 
+    --input-background-color: var(--background-color3);
+	--input-text-color: #ccc;
+	--input-border-color: var(--background-color4);
+}
+
+.theme-super-dark {
+    --main-hue: 222;
+	--main-saturation: 18%;
+	--value-base: 5%;
+	--value-step: 5%;
+	--background-color1: hsl(var(--main-hue), var(--main-saturation), var(--value-base));
+	--background-color2: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) + (1 * var(--value-step))));
+	--background-color3: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) + (2 * var(--value-step))));
+	--background-color4: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) + (1.4 * var(--value-step))));
+    
+    --primary-button-border: none;
+
+    --button-color: var(--accent-color);
+    --button-border: none;
+    
+    --input-border-size: 0px; 
+    --input-background-color: var(--background-color3);
+	--input-text-color: #ccc;
+	--input-border-color: var(--background-color4);
+}
+
+.theme-wild {
+    --main-hue: 128;
+	--main-saturation: 18%;
+	--value-base: 20%;
+	--value-step: 5%;
+	--background-color1: hsl(var(--main-hue), var(--main-saturation), var(--value-base));
+	--background-color2: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (1 * var(--value-step))));
+	--background-color3: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (2 * var(--value-step))));
+	--background-color4: hsl(var(--main-hue), var(--main-saturation), calc(var(--value-base) - (3 * var(--value-step))));
+    
+    --accent-hue: 212;
+    --primary-button-border: none;
+
+    --button-color: var(--accent-color);
+    --button-border: none;
+    
+    --input-border-size: 1px; 
+    --input-background-color: hsl(222, var(--main-saturation), calc(var(--value-base) - (2 * var(--value-step))));
+	--input-text-color: red;
+	--input-border-color: green;
+}

ui/media/js/auto-save.js

@@ -0,0 +1,299 @@
+// Saving settings
+let saveSettingsConfigTable = document.getElementById("save-settings-config-table")
+let saveSettingsConfigOverlay = document.getElementById("save-settings-config")
+let resetImageSettingsButton = document.getElementById("reset-image-settings")
+
+const SETTINGS_KEY = "user_settings_v2"
+
+const SETTINGS = {} // key=id. dict initialized in initSettings. { element, default, value, ignore }
+const SETTINGS_IDS_LIST = [
+    "prompt",
+    "seed",
+    "random_seed",
+    "num_outputs_total",
+    "num_outputs_parallel",
+    "stable_diffusion_model",
+    "vae_model",
+    "sampler",
+    "width",
+    "height",
+    "num_inference_steps",
+    "guidance_scale",
+    "prompt_strength",
+    "output_format",
+    "negative_prompt",
+    "stream_image_progress",
+    "use_face_correction",
+    "use_upscale",
+    "show_only_filtered_image",
+    "upscale_model",
+    "preview-image",
+    "modifier-card-size-slider",
+    "theme",
+    "save_to_disk",
+    "diskPath",
+    "sound_toggle",
+    "turbo",
+    "use_full_precision",
+    "auto_save_settings"
+]
+
+const IGNORE_BY_DEFAULT = [
+    "prompt"
+]
+
+const SETTINGS_SECTIONS = [ // gets the "keys" property filled in with an ordered list of settings in this section via initSettings
+    { id: "editor-inputs",   name: "Prompt" },
+    { id: "editor-settings", name: "Image Settings" },
+    { id: "system-settings", name: "System Settings" },
+    { id: "container",       name: "Other" }
+]
+
+async function initSettings() {
+    SETTINGS_IDS_LIST.forEach(id => {
+        var element = document.getElementById(id)
+        if (!element) {
+            console.error(`Missing settings element ${id}`)
+        }
+        SETTINGS[id] = {
+            key: id,
+            element: element,
+            label: getSettingLabel(element),
+            default: getSetting(element),
+            value: getSetting(element),
+            ignore: IGNORE_BY_DEFAULT.includes(id)
+        }
+        element.addEventListener("input", settingChangeHandler)
+        element.addEventListener("change", settingChangeHandler)
+    })
+    var unsorted_settings_ids = [...SETTINGS_IDS_LIST]
+    SETTINGS_SECTIONS.forEach(section => {
+        var name = section.name
+        var element = document.getElementById(section.id)
+        var unsorted_ids = unsorted_settings_ids.map(id => `#${id}`).join(",")
+        var children = unsorted_ids == "" ? [] : Array.from(element.querySelectorAll(unsorted_ids));
+        section.keys = []
+        children.forEach(e => {
+            section.keys.push(e.id)
+        })
+        unsorted_settings_ids = unsorted_settings_ids.filter(id => children.find(e => e.id == id) == undefined)
+    })
+    loadSettings()
+}
+
+function getSetting(element) {
+    if (typeof element === "string" || element instanceof String) {
+        element = SETTINGS[element].element
+    }
+    if (element.type == "checkbox") {
+        return element.checked
+    }
+    return element.value
+}
+function setSetting(element, value) {
+    if (typeof element === "string" || element instanceof String) {
+        element = SETTINGS[element].element
+    }
+    SETTINGS[element.id].value = value
+    if (getSetting(element) == value) {
+        return // no setting necessary
+    }
+    if (element.type == "checkbox") {
+        element.checked = value
+    }
+    else {
+        element.value = value
+    }
+    element.dispatchEvent(new Event("input"))
+    element.dispatchEvent(new Event("change"))
+}
+
+function saveSettings() {
+    var saved_settings = Object.values(SETTINGS).map(setting => {
+        return {
+            key: setting.key,
+            value: setting.value,
+            ignore: setting.ignore
+        }
+    })
+    localStorage.setItem(SETTINGS_KEY, JSON.stringify(saved_settings))
+}
+
+var CURRENTLY_LOADING_SETTINGS = false
+function loadSettings() {
+    var saved_settings_text = localStorage.getItem(SETTINGS_KEY)
+    if (saved_settings_text) {
+        var saved_settings = JSON.parse(saved_settings_text)
+        if (saved_settings.find(s => s.key == "auto_save_settings").value == false) {
+            setSetting("auto_save_settings", false)
+            return
+        }
+        CURRENTLY_LOADING_SETTINGS = true
+        saved_settings.forEach(saved_setting => {
+            var setting = SETTINGS[saved_setting.key]
+            if (!setting) {
+                console.warn(`Attempted to load setting ${saved_setting.key}, but no setting found`);
+                return null;
+            }
+            setting.ignore = saved_setting.ignore
+            if (!setting.ignore) {
+                setting.value = saved_setting.value
+                setSetting(setting.element, setting.value)
+            }
+        })
+        CURRENTLY_LOADING_SETTINGS = false
+    }
+    else {
+        CURRENTLY_LOADING_SETTINGS = true
+        tryLoadOldSettings();
+        CURRENTLY_LOADING_SETTINGS = false
+        saveSettings()
+    }
+}
+
+function loadDefaultSettingsSection(section_id) {
+    CURRENTLY_LOADING_SETTINGS = true
+    var section = SETTINGS_SECTIONS.find(s => s.id == section_id);
+    section.keys.forEach(key => {
+        var setting = SETTINGS[key];
+        setting.value = setting.default
+        setSetting(setting.element, setting.value)
+    })
+    CURRENTLY_LOADING_SETTINGS = false
+    saveSettings()
+}
+
+function settingChangeHandler(event) {
+    if (!CURRENTLY_LOADING_SETTINGS) {
+        var element = event.target
+        var value = getSetting(element)
+        if (value != SETTINGS[element.id].value) {
+            SETTINGS[element.id].value = value
+            saveSettings()
+        }
+    }
+}
+
+function getSettingLabel(element) {
+    var labelElement = document.querySelector(`label[for='${element.id}']`)
+    var label = labelElement?.innerText || element.id
+    var truncate_length = 30
+    if (label.includes(" (")) {
+        label = label.substring(0, label.indexOf(" ("))
+    }
+    if (label.length > truncate_length) {
+        label = label.substring(0, truncate_length - 3) + "..."
+    }
+    label = label.replace("➕", "")
+    label = label.replace("➖", "")
+    return label
+}
+
+function fillSaveSettingsConfigTable() {
+    saveSettingsConfigTable.textContent = ""
+    SETTINGS_SECTIONS.forEach(section => {
+        var section_row = `<tr><th>${section.name}</th><td></td></tr>`
+        saveSettingsConfigTable.insertAdjacentHTML("beforeend", section_row)
+        section.keys.forEach(key => {
+            var setting = SETTINGS[key]
+            var element = setting.element
+            var checkbox_id = `shouldsave_${element.id}`
+            var is_checked = setting.ignore ? "" : "checked"
+            var value = setting.value
+            var value_truncate_length = 30
+            if ((typeof value === "string" || value instanceof String) && value.length > value_truncate_length) {
+                value = value.substring(0, value_truncate_length - 3) + "..."
+            }
+            var newrow = `<tr><td><label for="${checkbox_id}">${setting.label}</label></td><td><input id="${checkbox_id}" name="${checkbox_id}" ${is_checked} type="checkbox" ></td><td><small>(${value})</small></td></tr>`
+            saveSettingsConfigTable.insertAdjacentHTML("beforeend", newrow)
+            var checkbox = document.getElementById(checkbox_id)
+            checkbox.addEventListener("input", event => {
+                setting.ignore = !checkbox.checked
+                saveSettings()
+            })
+        })
+    })
+}
+
+// configureSettingsSaveBtn
+
+
+
+
+var autoSaveSettings = document.getElementById("auto_save_settings")
+var configSettingsButton = document.createElement("button")
+configSettingsButton.textContent = "Configure"
+configSettingsButton.style.margin = "0px 5px"
+autoSaveSettings.insertAdjacentElement("afterend", configSettingsButton)
+autoSaveSettings.addEventListener("change", () => {
+    configSettingsButton.style.display = autoSaveSettings.checked ? "block" : "none"
+})
+configSettingsButton.addEventListener('click', () => {
+    fillSaveSettingsConfigTable()
+    saveSettingsConfigOverlay.classList.add("active")
+})
+resetImageSettingsButton.addEventListener('click', event => {
+    loadDefaultSettingsSection("editor-settings");
+    event.stopPropagation()
+})
+
+
+function tryLoadOldSettings() {
+    console.log("Loading old user settings")
+    // load v1 auto-save.js settings
+    var old_map = {
+        "guidance_scale_slider": "guidance_scale",
+        "prompt_strength_slider": "prompt_strength"
+    }
+    var settings_key_v1 = "user_settings"
+    var saved_settings_text = localStorage.getItem(settings_key_v1)
+    if (saved_settings_text) {
+        var saved_settings = JSON.parse(saved_settings_text)
+        Object.keys(saved_settings.should_save).forEach(key => {
+            key = key in old_map ? old_map[key] : key
+            SETTINGS[key].ignore = !saved_settings.should_save[key]
+        });
+        Object.keys(saved_settings.values).forEach(key => {
+            key = key in old_map ? old_map[key] : key
+            var setting = SETTINGS[key]
+            if (!setting.ignore) {
+                setting.value = saved_settings.values[key]
+                setSetting(setting.element, setting.value)
+            }
+        });
+        localStorage.removeItem(settings_key_v1)
+    }
+
+    // load old individually stored items
+    var individual_settings_map = { // maps old localStorage-key to new SETTINGS-key
+        "soundEnabled": "sound_toggle",
+        "saveToDisk": "save_to_disk",
+        "useCPU": "use_cpu",
+        "useFullPrecision": "use_full_precision",
+        "useTurboMode": "turbo",
+        "diskPath": "diskPath",
+        "useFaceCorrection": "use_face_correction",
+        "useUpscaling": "use_upscale",
+        "showOnlyFilteredImage": "show_only_filtered_image",
+        "streamImageProgress": "stream_image_progress",
+        "outputFormat": "output_format",
+        "autoSaveSettings": "auto_save_settings",
+    };
+    Object.keys(individual_settings_map).forEach(localStorageKey => {
+        var localStorageValue = localStorage.getItem(localStorageKey);
+        if (localStorageValue !== null) {
+            let key = individual_settings_map[localStorageKey]
+            var setting = SETTINGS[key]
+            if (!setting) {
+                console.warn(`Attempted to map old setting ${key}, but no setting found`);
+                return null;
+            }
+            if (setting.element.type == "checkbox" && (typeof localStorageValue === "string" || localStorageValue instanceof String)) {
+                localStorageValue = localStorageValue == "true"
+            }
+            setting.value = localStorageValue
+            setSetting(setting.element, setting.value)
+            localStorage.removeItem(localStorageKey);
+        }
+    })
+}

ui/media/js/dnd.js

@@ -0,0 +1,469 @@
+"use strict" // Opt in to a restricted variant of JavaScript
+
+const EXT_REGEX = /(?:\.([^.]+))?$/
+const TEXT_EXTENSIONS = ['txt', 'json']
+const IMAGE_EXTENSIONS = ['jpg', 'jpeg', 'png', 'bmp', 'tiff', 'tif', 'tga']
+
+function parseBoolean(stringValue) {
+    if (typeof stringValue === 'boolean') {
+        return stringValue
+    }
+    if (typeof stringValue === 'number') {
+        return stringValue !== 0
+    }
+    if (typeof stringValue !== 'string') {
+        return false
+    }
+    switch(stringValue?.toLowerCase()?.trim()) {
+        case "true":
+        case "yes":
+        case "on":
+        case "1":
+          return true;
+
+        case "false":
+        case "no":
+        case "off":
+        case "0":
+        case null:
+        case undefined:
+          return false;
+    }
+    try {
+        return Boolean(JSON.parse(stringValue));
+    } catch {
+        return Boolean(stringValue)
+    }
+}
+
+const TASK_MAPPING = {
+    prompt: { name: 'Prompt',
+        setUI: (prompt) => {
+            promptField.value = prompt
+        },
+        readUI: () => promptField.value,
+        parse: (val) => val
+    },
+    negative_prompt: { name: 'Negative Prompt',
+        setUI: (negative_prompt) => {
+            negativePromptField.value = negative_prompt
+        },
+        readUI: () => negativePromptField.value,
+        parse: (val) => val
+    },
+    width: { name: 'Width',
+        setUI: (width) => {
+            const oldVal = widthField.value
+            widthField.value = width
+            if (!widthField.value) {
+                widthField.value = oldVal
+            }
+        },
+        readUI: () => parseInt(widthField.value),
+        parse: (val) => parseInt(val)
+    },
+    height: { name: 'Height',
+        setUI: (height) => {
+            const oldVal = heightField.value
+            heightField.value = height
+            if (!heightField.value) {
+                heightField.value = oldVal
+            }
+        },
+        readUI: () => parseInt(heightField.value),
+        parse: (val) => parseInt(val)
+    },
+    seed: { name: 'Seed',
+        setUI: (seed) => {
+            if (!seed) {
+                randomSeedField.checked = true
+                seedField.disabled = true
+                return
+            }
+            randomSeedField.checked = false
+            seedField.disabled = false
+            seedField.value = seed
+        },
+        readUI: () => (randomSeedField.checked ? Math.floor(Math.random() * 10000000) : parseInt(seedField.value)),
+        parse: (val) => parseInt(val)
+    },
+    num_inference_steps: { name: 'Steps',
+        setUI: (num_inference_steps) => {
+            numInferenceStepsField.value = num_inference_steps
+        },
+        readUI: () => parseInt(numInferenceStepsField.value),
+        parse: (val) => parseInt(val)
+    },
+    guidance_scale: { name: 'Guidance Scale',
+        setUI: (guidance_scale) => {
+            guidanceScaleField.value = guidance_scale
+            updateGuidanceScaleSlider()
+        },
+        readUI: () => parseFloat(guidanceScaleField.value),
+        parse: (val) => parseFloat(val)
+    },
+    prompt_strength: { name: 'Prompt Strength',
+        setUI: (prompt_strength) => {
+            promptStrengthField.value = prompt_strength
+            updatePromptStrengthSlider()
+        },
+        readUI: () => parseFloat(promptStrengthField.value),
+        parse: (val) => parseFloat(val)
+    },
+
+    init_image:  { name: 'Initial Image',
+        setUI: (init_image) => {
+            initImagePreview.src = init_image
+        },
+        readUI: () => initImagePreview.src,
+        parse: (val) => val
+    },
+    mask:  { name: 'Mask',
+        setUI: (mask) => {
+            inpaintingEditor.setImg(mask)
+            maskSetting.checked = Boolean(mask)
+        },
+        readUI: () => (maskSetting.checked ? inpaintingEditor.getImg() : undefined),
+        parse: (val) => val
+    },
+
+    use_face_correction: { name: 'Use Face Correction',
+        setUI: (use_face_correction) => {
+            useFaceCorrectionField.checked = parseBoolean(use_face_correction)
+        },
+        readUI: () => useFaceCorrectionField.checked,
+        parse: (val) => parseBoolean(val)
+    },
+    use_upscale: { name: 'Use Upscaling',
+        setUI: (use_upscale) => {
+            const oldVal = upscaleModelField.value
+            upscaleModelField.value = use_upscale
+            if (upscaleModelField.value) { // Is a valid value for the field.
+                useUpscalingField.checked = true
+                upscaleModelField.disabled = false
+            } else { // Not a valid value, restore the old value and disable the filter.
+                upscaleModelField.disabled = true
+                upscaleModelField.value = oldVal
+                useUpscalingField.checked = false
+            }
+        },
+        readUI: () => (useUpscalingField.checked ? upscaleModelField.value : undefined),
+        parse: (val) => val
+    },
+    sampler: { name: 'Sampler',
+        setUI: (sampler) => {
+            samplerField.value = sampler
+        },
+        readUI: () => samplerField.value,
+        parse: (val) => val
+    },
+    use_stable_diffusion_model: { name: 'Stable Diffusion model',
+        setUI: (use_stable_diffusion_model) => {
+            const oldVal = stableDiffusionModelField.value
+
+            let pathIdx = use_stable_diffusion_model.lastIndexOf('/') // Linux, Mac paths
+            if (pathIdx < 0) {
+                pathIdx = use_stable_diffusion_model.lastIndexOf('\\') // Windows paths.
+            }
+            if (pathIdx >= 0) {
+                use_stable_diffusion_model = use_stable_diffusion_model.slice(pathIdx + 1)
+            }
+            const modelExt = '.ckpt'
+            if (use_stable_diffusion_model.endsWith(modelExt)) {
+                use_stable_diffusion_model = use_stable_diffusion_model.slice(0, use_stable_diffusion_model.length - modelExt.length)
+            }
+
+            stableDiffusionModelField.value = use_stable_diffusion_model
+
+            if (!stableDiffusionModelField.value) {
+                stableDiffusionModelField.value = oldVal
+            }
+        },
+        readUI: () => stableDiffusionModelField.value,
+        parse: (val) => val
+    },
+
+    numOutputsParallel: { name: 'Parallel Images',
+        setUI: (numOutputsParallel) => {
+            numOutputsParallelField.value = numOutputsParallel
+        },
+        readUI: () => parseInt(numOutputsParallelField.value),
+        parse: (val) => val
+    },
+
+    use_cpu: { name: 'Use CPU',
+        setUI: (use_cpu) => {
+            useCPUField.checked = use_cpu
+        },
+        readUI: () => useCPUField.checked,
+        parse: (val) => val
+    },
+    turbo: { name: 'Turbo',
+        setUI: (turbo) => {
+            turboField.checked = turbo
+        },
+        readUI: () => turboField.checked,
+        parse: (val) => Boolean(val)
+    },
+    use_full_precision: { name: 'Use Full Precision',
+        setUI: (use_full_precision) => {
+            useFullPrecisionField.checked = use_full_precision
+        },
+        readUI: () => useFullPrecisionField.checked,
+        parse: (val) => Boolean(val)
+    },
+
+    stream_image_progress: { name: 'Stream Image Progress',
+        setUI: (stream_image_progress) => {
+            streamImageProgressField.checked = (parseInt(numOutputsTotalField.value) > 50 ? false : stream_image_progress)
+        },
+        readUI: () => streamImageProgressField.checked,
+        parse: (val) => Boolean(val)
+    },
+    show_only_filtered_image: { name: 'Show only the corrected/upscaled image',
+        setUI: (show_only_filtered_image) => {
+            showOnlyFilteredImageField.checked = show_only_filtered_image
+        },
+        readUI: () => showOnlyFilteredImageField.checked,
+        parse: (val) => Boolean(val)
+    },
+    output_format: { name: 'Output Format',
+        setUI: (output_format) => {
+            outputFormatField.value = output_format
+        },
+        readUI: () => outputFormatField.value,
+        parse: (val) => val
+    },
+    save_to_disk_path: { name: 'Save to disk path',
+        setUI: (save_to_disk_path) => {
+            saveToDiskField.checked = Boolean(save_to_disk_path)
+            diskPathField.value = save_to_disk_path
+        },
+        readUI: () => diskPathField.value,
+        parse: (val) => val
+    }
+}
+function restoreTaskToUI(task) {
+    if ('numOutputsTotal' in task) {
+        numOutputsTotalField.value = task.numOutputsTotal
+    }
+    if ('seed' in task) {
+        randomSeedField.checked = false
+        seedField.value = task.seed
+    }
+    if (!('reqBody' in task)) {
+        return
+    }
+    for (const key in TASK_MAPPING) {
+        if (key in task.reqBody) {
+            TASK_MAPPING[key].setUI(task.reqBody[key])
+        }
+    }
+}
+function readUI() {
+    const reqBody = {}
+    for (const key in TASK_MAPPING) {
+        reqBody[key] = TASK_MAPPING[key].readUI()
+    }
+    return {
+        'numOutputsTotal': parseInt(numOutputsTotalField.value),
+        'seed': TASK_MAPPING['seed'].readUI(),
+        'reqBody': reqBody
+    }
+}
+
+const TASK_TEXT_MAPPING = {
+    width: 'Width',
+    height: 'Height',
+    seed: 'Seed',
+    num_inference_steps: 'Steps',
+    guidance_scale: 'Guidance Scale',
+    prompt_strength: 'Prompt Strength',
+    use_face_correction: 'Use Face Correction',
+    use_upscale: 'Use Upscaling',
+    sampler: 'Sampler',
+    negative_prompt: 'Negative Prompt',
+    use_stable_diffusion_model: 'Stable Diffusion model'
+}
+const afterPromptRe = /^\s*Width\s*:\s*\d+\s*(?:\r\n|\r|\n)+\s*Height\s*:\s*\d+\s*(\r\n|\r|\n)+Seed\s*:\s*\d+\s*$/igm
+function parseTaskFromText(str) {
+    const taskReqBody = {}
+
+    // Prompt
+    afterPromptRe.lastIndex = 0
+    const match = afterPromptRe.exec(str)
+    if (match) {
+        let prompt = str.slice(0, match.index)
+        str = str.slice(prompt.length)
+        taskReqBody.prompt = prompt.trim()
+        console.log('Prompt:', taskReqBody.prompt)
+    }
+    for (const key in TASK_TEXT_MAPPING) {
+        const name = TASK_TEXT_MAPPING[key];
+        let val = undefined
+
+        const reName = new RegExp(`${name}\\ *:\\ *(.*)(?:\\r\\n|\\r|\\n)*`, 'igm')
+        const match = reName.exec(str);
+        if (match) {
+            str = str.slice(0, match.index) + str.slice(match.index + match[0].length)
+            val = match[1]
+        }
+        if (val !== undefined) {
+            taskReqBody[key] = TASK_MAPPING[key].parse(val.trim())
+            console.log(TASK_MAPPING[key].name + ':', taskReqBody[key])
+            if (!str) {
+                break
+            }
+        }
+    }
+    if (Object.keys(taskReqBody).length <= 0) {
+        return undefined
+    }
+    const task = { reqBody: taskReqBody }
+    if ('seed' in taskReqBody) {
+        task.seed = taskReqBody.seed
+    }
+    return task
+}
+
+async function readFile(file, i) {
+    const fileContent = (await file.text()).trim()
+
+    // JSON File.
+    if (fileContent.startsWith('{') && fileContent.endsWith('}')) {
+        try {
+            const task = JSON.parse(fileContent)
+            restoreTaskToUI(task)
+        } catch (e) {
+            console.warn(`file[${i}]:${file.name} - File couldn't be parsed.`, e)
+        }
+        return
+    }
+
+    // Normal txt file.
+    const task = parseTaskFromText(fileContent)
+    if (task) {
+        restoreTaskToUI(task)
+    } else {
+        console.warn(`file[${i}]:${file.name} - File couldn't be parsed.`)
+    }
+}
+
+function dropHandler(ev) {
+    console.log('Content dropped...')
+    let items = []
+
+    if (ev?.dataTransfer?.items) { // Use DataTransferItemList interface
+        items = Array.from(ev.dataTransfer.items)
+        items = items.filter(item => item.kind === 'file')
+        items = items.map(item => item.getAsFile())
+    } else if (ev?.dataTransfer?.files) { // Use DataTransfer interface
+        items = Array.from(ev.dataTransfer.files)
+    }
+
+    items.forEach(item => {item.file_ext = EXT_REGEX.exec(item.name.toLowerCase())[1]})
+
+    let text_items = items.filter(item => TEXT_EXTENSIONS.includes(item.file_ext))
+    let image_items = items.filter(item => IMAGE_EXTENSIONS.includes(item.file_ext))
+
+    if (image_items.length > 0 && ev.target == initImageSelector) {
+        return // let the event bubble up, so that the Init Image filepicker can receive this
+    }
+
+    ev.preventDefault() // Prevent default behavior (Prevent file/content from being opened)
+    text_items.forEach(readFile)
+}
+function dragOverHandler(ev) {
+    console.log('Content in drop zone')
+
+    // Prevent default behavior (Prevent file/content from being opened)
+    ev.preventDefault()
+
+    ev.dataTransfer.dropEffect = "copy"
+
+    let img = new Image()
+    img.src = location.host + '/media/images/favicon-32x32.png'
+    ev.dataTransfer.setDragImage(img, 16, 16)
+}
+
+document.addEventListener("drop", dropHandler)
+document.addEventListener("dragover", dragOverHandler)
+
+const TASK_REQ_NO_EXPORT = [
+    "use_cpu",
+    "turbo",
+    "use_full_precision",
+    "save_to_disk_path"
+]
+
+// Retrieve clipboard content and try to parse it 
+async function pasteFromClipboard() {
+    //const text = await navigator.clipboard.readText()
+    let text = await navigator.clipboard.readText();
+    text=text.trim();
+    if (text.startsWith('{') && text.endsWith('}')) {
+        try {
+            const task = JSON.parse(text)
+            restoreTaskToUI(task)
+        } catch (e) {
+            console.warn(`Clipboard JSON couldn't be parsed.`, e)
+        }
+        return
+    }
+    // Normal txt file.
+    const task = parseTaskFromText(text)
+    if (task) {
+        restoreTaskToUI(task)
+    } else {
+        console.warn(`Clipboard content - File couldn't be parsed.`)
+    }
+}
+
+// Adds a copy and a paste icon if the browser grants permission to write to clipboard.
+function checkWriteToClipboardPermission (result) {
+    if (result.state == "granted" || result.state == "prompt") {
+        const resetSettings = document.getElementById('reset-image-settings')
+
+        // COPY ICON
+        const copyIcon = document.createElement('i')
+        copyIcon.className = 'fa-solid fa-clipboard section-button'
+        copyIcon.innerHTML = `<span class="simple-tooltip right">Copy Image Settings</span>`
+        copyIcon.addEventListener('click', (event) => {
+            event.stopPropagation()
+	    // Add css class 'active'
+	    copyIcon.classList.add('active')
+	    // In 1000 ms remove the 'active' class
+	    asyncDelay(1000).then(() => copyIcon.classList.remove('active'))
+            const uiState = readUI()
+            TASK_REQ_NO_EXPORT.forEach((key) => delete uiState.reqBody[key])
+            if (uiState.reqBody.init_image && !IMAGE_REGEX.test(uiState.reqBody.init_image)) {
+                delete uiState.reqBody.init_image
+                delete uiState.reqBody.prompt_strength
+            }
+            navigator.clipboard.writeText(JSON.stringify(uiState, undefined, 4))
+        })
+        resetSettings.parentNode.insertBefore(copyIcon, resetSettings)
+
+        // PASTE ICON
+        const pasteIcon = document.createElement('i')
+        pasteIcon.className = 'fa-solid fa-paste section-button'
+        pasteIcon.innerHTML = `<span class="simple-tooltip right">Paste Image Settings</span>`
+        pasteIcon.addEventListener('click', (event) => {
+            event.stopPropagation()
+	    // Add css class 'active'
+	    pasteIcon.classList.add('active')
+	    // In 1000 ms remove the 'active' class
+	    asyncDelay(1000).then(() => pasteIcon.classList.remove('active'))
+            pasteFromClipboard()
+        })
+        resetSettings.parentNode.insertBefore(pasteIcon, resetSettings)
+    }
+}
+
+// Determine which access we have to the clipboard. Clipboard access is only available  on localhost or via TLS.
+navigator.permissions.query({ name: "clipboard-write" }).then(checkWriteToClipboardPermission, (e) => {
+    if (e instanceof TypeError && typeof navigator?.clipboard?.writeText === 'function') {
+        // Fix for firefox https://bugzilla.mozilla.org/show_bug.cgi?id=1560373
+        checkWriteToClipboardPermission({state:"granted"})
+    }
+})

ui/media/js/image-modifiers.js

@@ -0,0 +1,286 @@
+let activeTags = []
+let modifiers = []
+let customModifiersGroupElement = undefined
+
+let editorModifierEntries = document.querySelector('#editor-modifiers-entries')
+let editorModifierTagsList = document.querySelector('#editor-inputs-tags-list')
+let editorTagsContainer = document.querySelector('#editor-inputs-tags-container')
+let modifierCardSizeSlider = document.querySelector('#modifier-card-size-slider')
+let previewImageField = document.querySelector('#preview-image')
+let modifierSettingsBtn = document.querySelector('#modifier-settings-btn')
+let modifierSettingsOverlay = document.querySelector('#modifier-settings-config')
+let customModifiersTextBox = document.querySelector('#custom-modifiers-input')
+let customModifierEntriesToolbar = document.querySelector('#editor-modifiers-entries-toolbar')
+
+const modifierThumbnailPath = 'media/modifier-thumbnails'
+const activeCardClass = 'modifier-card-active'
+const CUSTOM_MODIFIERS_KEY = "customModifiers"
+
+function createModifierCard(name, previews) {
+    const modifierCard = document.createElement('div')
+    modifierCard.className = 'modifier-card'
+    modifierCard.innerHTML = `
+    <div class="modifier-card-overlay"></div>
+    <div class="modifier-card-image-container">
+        <div class="modifier-card-image-overlay">+</div>
+        <p class="modifier-card-error-label"></p>
+        <img onerror="this.remove()" alt="Modifier Image" class="modifier-card-image">
+    </div>
+    <div class="modifier-card-container">
+        <div class="modifier-card-label"><p></p></div>
+    </div>`
+
+    const image = modifierCard.querySelector('.modifier-card-image')
+    const errorText =  modifierCard.querySelector('.modifier-card-error-label')
+    const label = modifierCard.querySelector('.modifier-card-label')
+
+    errorText.innerText = 'No Image'
+
+    if (typeof previews == 'object') {
+        image.src = previews[0]; // portrait
+        image.setAttribute('preview-type', 'portrait')
+    } else {
+        image.remove()
+    }
+
+    const maxLabelLength = 30
+    const nameWithoutBy = name.replace('by ', '')
+
+    if(nameWithoutBy.length <= maxLabelLength) {
+        label.querySelector('p').innerText = nameWithoutBy
+    } else {
+        const tooltipText = document.createElement('span')
+        tooltipText.className = 'tooltip-text'
+        tooltipText.innerText = name
+
+        label.classList.add('tooltip')
+        label.appendChild(tooltipText)
+
+        label.querySelector('p').innerText = nameWithoutBy.substring(0, maxLabelLength) + '...'
+    }
+
+    return modifierCard
+}
+
+function createModifierGroup(modifierGroup, initiallyExpanded) {
+    const title = modifierGroup.category
+    const modifiers = modifierGroup.modifiers
+
+    const titleEl = document.createElement('h5')
+    titleEl.className = 'collapsible'
+    titleEl.innerText = title
+
+    const modifiersEl = document.createElement('div')
+    modifiersEl.classList.add('collapsible-content', 'editor-modifiers-leaf')
+
+    if (initiallyExpanded === true) {
+        titleEl.className += ' active'
+    }
+
+    modifiers.forEach(modObj => {
+        const modifierName = modObj.modifier
+        const modifierPreviews = modObj?.previews?.map(preview => `${modifierThumbnailPath}/${preview.path}`)
+
+        const modifierCard = createModifierCard(modifierName, modifierPreviews)
+
+        if(typeof modifierCard == 'object') {
+            modifiersEl.appendChild(modifierCard)
+
+            modifierCard.addEventListener('click', () => {
+                if (activeTags.map(x => x.name).includes(modifierName)) {
+                    // remove modifier from active array
+                    activeTags = activeTags.filter(x => x.name != modifierName)
+                    modifierCard.classList.remove(activeCardClass)
+
+                    modifierCard.querySelector('.modifier-card-image-overlay').innerText = '+'
+                } else {
+                    // add modifier to active array
+                    activeTags.push({
+                        'name': modifierName,
+                        'element': modifierCard.cloneNode(true),
+                        'originElement': modifierCard,
+                        'previews': modifierPreviews
+                    })
+
+                    modifierCard.classList.add(activeCardClass)
+
+                    modifierCard.querySelector('.modifier-card-image-overlay').innerText = '-'
+                }
+
+                refreshTagsList()
+            })
+        }
+    })
+
+    let brk = document.createElement('br')
+    brk.style.clear = 'both'
+    modifiersEl.appendChild(brk)
+
+    let e = document.createElement('div')
+    e.appendChild(titleEl)
+    e.appendChild(modifiersEl)
+
+    editorModifierEntries.insertBefore(e, customModifierEntriesToolbar.nextSibling)
+
+    return e
+}
+
+async function loadModifiers() {
+    try {
+        let res = await fetch('/get/modifiers')
+        if (res.status === 200) {
+            res = await res.json()
+
+            modifiers = res; // update global variable
+
+            res.reverse()
+
+            res.forEach((modifierGroup, idx) => {
+                createModifierGroup(modifierGroup, idx === res.length - 1)
+            })
+
+            createCollapsibles(editorModifierEntries)
+        }
+    } catch (e) {
+        console.log('error fetching modifiers', e)
+    }
+
+    loadCustomModifiers()
+}
+
+function refreshTagsList() {
+    editorModifierTagsList.innerHTML = ''
+
+    if (activeTags.length == 0) {
+        editorTagsContainer.style.display = 'none'
+        return
+    } else {
+        editorTagsContainer.style.display = 'block'
+    }
+
+    activeTags.forEach((tag, index) => {
+        tag.element.querySelector('.modifier-card-image-overlay').innerText = '-'
+        tag.element.classList.add('modifier-card-tiny')
+
+        editorModifierTagsList.appendChild(tag.element)
+
+        tag.element.addEventListener('click', () => {
+            let idx = activeTags.indexOf(tag)
+
+            if (idx !== -1) {
+                activeTags[idx].originElement.classList.remove(activeCardClass)
+                activeTags[idx].originElement.querySelector('.modifier-card-image-overlay').innerText = '+'
+
+                activeTags.splice(idx, 1)
+                refreshTagsList()
+            }
+        })
+    })
+
+    let brk = document.createElement('br')
+    brk.style.clear = 'both'
+    editorModifierTagsList.appendChild(brk)
+}
+
+function changePreviewImages(val) {
+    const previewImages = document.querySelectorAll('.modifier-card-image-container img')
+
+    let previewArr = []
+
+    modifiers.map(x => x.modifiers).forEach(x => previewArr.push(...x.map(m => m.previews)))
+    
+    previewArr = previewArr.map(x => {
+        let obj = {}
+
+        x.forEach(preview => {
+            obj[preview.name] = preview.path
+        })
+        
+        return obj
+    })
+
+    previewImages.forEach(previewImage => {
+        const currentPreviewType = previewImage.getAttribute('preview-type')
+        const relativePreviewPath = previewImage.src.split(modifierThumbnailPath + '/').pop()
+
+        const previews = previewArr.find(preview => relativePreviewPath == preview[currentPreviewType])
+
+        if(typeof previews == 'object') {
+            let preview = null
+
+            if (val == 'portrait') {
+                preview = previews.portrait
+            }
+            else if (val == 'landscape') {
+                preview = previews.landscape
+            }
+
+            if(preview != null) {
+                previewImage.src = `${modifierThumbnailPath}/${preview}`
+                previewImage.setAttribute('preview-type', val)
+            }
+        }
+    })
+}
+
+function resizeModifierCards(val) {
+    const cardSizePrefix = 'modifier-card-size_'
+    const modifierCardClass = 'modifier-card'
+
+    const modifierCards = document.querySelectorAll(`.${modifierCardClass}`)
+    const cardSize = n => `${cardSizePrefix}${n}`
+
+    modifierCards.forEach(card => {
+        // remove existing size classes
+        const classes = card.className.split(' ').filter(c => !c.startsWith(cardSizePrefix))
+        card.className = classes.join(' ').trim()
+
+        if(val != 0) {
+            card.classList.add(cardSize(val))
+        }
+    })
+}
+
+modifierCardSizeSlider.onchange = () => resizeModifierCards(modifierCardSizeSlider.value)
+previewImageField.onchange = () => changePreviewImages(previewImageField.value)
+
+modifierSettingsBtn.addEventListener('click', function(e) {
+    modifierSettingsOverlay.classList.add("active")
+    e.stopPropagation()
+})
+
+function saveCustomModifiers() {
+    localStorage.setItem(CUSTOM_MODIFIERS_KEY, customModifiersTextBox.value.trim())
+
+    loadCustomModifiers()
+}
+
+function loadCustomModifiers() {
+    let customModifiers = localStorage.getItem(CUSTOM_MODIFIERS_KEY, '')
+    customModifiersTextBox.value = customModifiers
+
+    if (customModifiersGroupElement !== undefined) {
+        customModifiersGroupElement.remove()
+    }
+
+    if (customModifiers && customModifiers.trim() !== '') {
+        customModifiers = customModifiers.split('\n')
+        customModifiers = customModifiers.filter(m => m.trim() !== '')
+        customModifiers = customModifiers.map(function(m) {
+            return {
+                "modifier": m
+            }
+        })
+
+        let customGroup = {
+            'category': 'Custom Modifiers',
+            'modifiers': customModifiers
+        }
+
+        customModifiersGroupElement = createModifierGroup(customGroup, true)
+
+        createCollapsibles(customModifiersGroupElement)
+    }
+}
+
+customModifiersTextBox.addEventListener('change', saveCustomModifiers)

ui/media/js/inpainting-editor.js

@@ -0,0 +1,41 @@
+const INPAINTING_EDITOR_SIZE = 450
+
+let inpaintingEditorContainer = document.querySelector('#inpaintingEditor')
+let inpaintingEditor = new DrawingBoard.Board('inpaintingEditor', {
+    color: "#ffffff",
+    background: false,
+    size: 30,
+    webStorage: false,
+    controls: [{'DrawingMode': {'filler': false}}, 'Size', 'Navigation']
+})
+let inpaintingEditorCanvasBackground = document.querySelector('.drawing-board-canvas-wrapper')
+
+function resizeInpaintingEditor(widthValue, heightValue) {
+    if (widthValue === heightValue) {
+        widthValue = INPAINTING_EDITOR_SIZE
+        heightValue = INPAINTING_EDITOR_SIZE
+    } else if (widthValue > heightValue) {
+        heightValue = (heightValue / widthValue) * INPAINTING_EDITOR_SIZE
+        widthValue = INPAINTING_EDITOR_SIZE
+    } else {
+        widthValue = (widthValue / heightValue) * INPAINTING_EDITOR_SIZE
+        heightValue = INPAINTING_EDITOR_SIZE
+    }
+    if (inpaintingEditor.opts.aspectRatio === (widthValue / heightValue).toFixed(3)) {
+        // Same ratio, don't reset the canvas.
+        return
+    }
+    inpaintingEditor.opts.aspectRatio = (widthValue / heightValue).toFixed(3)
+
+    inpaintingEditorContainer.style.width = widthValue + 'px'
+    inpaintingEditorContainer.style.height = heightValue + 'px'
+    inpaintingEditor.opts.enlargeYourContainer = true
+
+    inpaintingEditor.opts.size = inpaintingEditor.ctx.lineWidth
+    inpaintingEditor.resize()
+
+    inpaintingEditor.ctx.lineCap = "round"
+    inpaintingEditor.ctx.lineJoin = "round"
+    inpaintingEditor.ctx.lineWidth = inpaintingEditor.opts.size
+    inpaintingEditor.setColor(inpaintingEditor.opts.color)
+}

ui/media/js/parameters.js

@@ -0,0 +1,330 @@
+/**
+ * Enum of parameter types
+ * @readonly
+ * @enum {string}
+ */
+ var ParameterType = {
+    checkbox: "checkbox",
+	select: "select",
+	select_multiple: "select_multiple",
+	custom: "custom",
+};
+
+/**
+ * JSDoc style
+ * @typedef {object} Parameter
+ * @property {string} id
+ * @property {ParameterType} type
+ * @property {string} label
+ * @property {?string} note
+ * @property {number|boolean|string} default
+ */
+
+
+/** @type {Array.<Parameter>} */
+var PARAMETERS = [
+	{
+		id: "theme",
+		type: ParameterType.select,
+		label: "Theme",
+		default: "theme-default",
+		options: [ // Note: options expanded dynamically
+			{
+				value: "theme-default",
+				label: "Default"
+			}
+		]
+	},
+	{
+		id: "save_to_disk",
+		type: ParameterType.checkbox,
+		label: "Auto-Save Images",
+		note: "automatically saves images to the specified location",
+		default: false,
+	},
+	{
+		id: "diskPath",
+		type: ParameterType.custom,
+		label: "Save Location",
+		render: (parameter) => {
+			return `<input id="${parameter.id}" name="${parameter.id}" size="30" disabled>`
+		}
+	},
+	{
+		id: "sound_toggle",
+		type: ParameterType.checkbox,
+		label: "Enable Sound",
+		note: "plays a sound on task completion",
+		default: true,
+	},
+	{
+		id: "ui_open_browser_on_start",
+		type: ParameterType.checkbox,
+		label: "Open browser on startup",
+		note: "starts the default browser on startup",
+		default: true,
+	},
+	{
+		id: "turbo",
+		type: ParameterType.checkbox,
+		label: "Turbo Mode",
+		default: true,
+		note: "generates images faster, but uses an additional 1 GB of GPU memory",
+	},
+	{
+		id: "use_cpu",
+		type: ParameterType.checkbox,
+		label: "Use CPU (not GPU)",
+		note: "warning: this will be *very* slow",
+		default: false,
+	},
+	{
+		id: "auto_pick_gpus",
+		type: ParameterType.checkbox,
+		label: "Automatically pick the GPUs (experimental)",
+		default: false,
+	},
+	{
+		id: "use_gpus",
+		type: ParameterType.select_multiple,
+		label: "GPUs to use (experimental)",
+		note: "to process in parallel",
+		default: false,
+	},
+	{
+		id: "use_full_precision",
+		type: ParameterType.checkbox,
+		label: "Use Full Precision",
+		note: "for GPU-only. warning: this will consume more VRAM",
+		default: false,
+	},
+	{
+		id: "auto_save_settings",
+		type: ParameterType.checkbox,
+		label: "Auto-Save Settings",
+		note: "restores settings on browser load",
+		default: true,
+	},
+	{
+		id: "use_beta_channel",
+		type: ParameterType.checkbox,
+		label: "🔥Beta channel",
+		note: "Get the latest features immediately (but could be less stable). Please restart the program after changing this.",
+		default: false,
+	},
+];
+
+function getParameterSettingsEntry(id) {
+	let parameter = PARAMETERS.filter(p => p.id === id)
+	if (parameter.length === 0) {
+		return
+	}
+	return parameter[0].settingsEntry
+}
+
+function getParameterElement(parameter) {
+	switch (parameter.type) {
+		case ParameterType.checkbox:
+			var is_checked = parameter.default ? " checked" : "";
+			return `<input id="${parameter.id}" name="${parameter.id}"${is_checked} type="checkbox">`
+		case ParameterType.select:
+		case ParameterType.select_multiple:
+			var options = (parameter.options || []).map(option => `<option value="${option.value}">${option.label}</option>`).join("")
+			var multiple = (parameter.type == ParameterType.select_multiple ? 'multiple' : '')
+			return `<select id="${parameter.id}" name="${parameter.id}" ${multiple}>${options}</select>`
+		case ParameterType.custom:
+			return parameter.render(parameter)
+		default:
+			console.error(`Invalid type for parameter ${parameter.id}`);
+			return "ERROR: Invalid Type"
+	}
+}
+
+let parametersTable = document.querySelector("#system-settings table")
+/* fill in the system settings popup table */
+function initParameters() {
+	PARAMETERS.forEach(parameter => {
+		var element = getParameterElement(parameter)
+		var note = parameter.note ? `<small>${parameter.note}</small>` : "";
+		var newrow = document.createElement('tr')
+		newrow.innerHTML = `
+			<td><label for="${parameter.id}">${parameter.label}</label></td>
+			<td><div>${element}${note}<div></td>`
+		parametersTable.appendChild(newrow)
+		parameter.settingsEntry = newrow
+	})
+}
+
+initParameters()
+
+let turboField = document.querySelector('#turbo')
+let useCPUField = document.querySelector('#use_cpu')
+let autoPickGPUsField = document.querySelector('#auto_pick_gpus')
+let useGPUsField = document.querySelector('#use_gpus')
+let useFullPrecisionField = document.querySelector('#use_full_precision')
+let saveToDiskField = document.querySelector('#save_to_disk')
+let diskPathField = document.querySelector('#diskPath')
+let useBetaChannelField = document.querySelector("#use_beta_channel")
+let uiOpenBrowserOnStartField = document.querySelector("#ui_open_browser_on_start")
+
+let saveSettingsBtn = document.querySelector('#save-system-settings-btn')
+
+async function changeAppConfig(configDelta) {
+    try {
+        let res = await fetch('/app_config', {
+            method: 'POST',
+            headers: {
+                'Content-Type': 'application/json'
+            },
+            body: JSON.stringify(configDelta)
+        })
+        res = await res.json()
+
+        console.log('set config status response', res)
+    } catch (e) {
+        console.log('set config status error', e)
+    }
+}
+
+async function getAppConfig() {
+    try {
+        let res = await fetch('/get/app_config')
+        const config = await res.json()
+
+        if (config.update_branch === 'beta') {
+            useBetaChannelField.checked = true
+        }
+        if (config.ui && config.ui.open_browser_on_start === false) {
+            uiOpenBrowserOnStartField.checked = false
+        }
+
+        console.log('get config status response', config)
+    } catch (e) {
+        console.log('get config status error', e)
+    }
+}
+
+saveToDiskField.addEventListener('change', function(e) {
+    diskPathField.disabled = !this.checked
+})
+
+function getCurrentRenderDeviceSelection() {
+    let selectedGPUs = $('#use_gpus').val()
+
+    if (useCPUField.checked && !autoPickGPUsField.checked) {
+        return 'cpu'
+    }
+    if (autoPickGPUsField.checked || selectedGPUs.length == 0) {
+        return 'auto'
+    }
+
+    return selectedGPUs.join(',')
+}
+
+useCPUField.addEventListener('click', function() {
+    let gpuSettingEntry = getParameterSettingsEntry('use_gpus')
+    let autoPickGPUSettingEntry = getParameterSettingsEntry('auto_pick_gpus')
+    if (this.checked) {
+        gpuSettingEntry.style.display = 'none'
+        autoPickGPUSettingEntry.style.display = 'none'
+        autoPickGPUsField.setAttribute('data-old-value', autoPickGPUsField.checked)
+        autoPickGPUsField.checked = false
+    } else if (useGPUsField.options.length >= MIN_GPUS_TO_SHOW_SELECTION) {
+        gpuSettingEntry.style.display = ''
+        autoPickGPUSettingEntry.style.display = ''
+        let oldVal = autoPickGPUsField.getAttribute('data-old-value')
+        if (oldVal === null || oldVal === undefined) { // the UI started with CPU selected by default
+            autoPickGPUsField.checked = true
+        } else {
+            autoPickGPUsField.checked = (oldVal === 'true')
+        }
+        gpuSettingEntry.style.display = (autoPickGPUsField.checked ? 'none' : '')
+    }
+})
+
+useGPUsField.addEventListener('click', function() {
+    let selectedGPUs = $('#use_gpus').val()
+    autoPickGPUsField.checked = (selectedGPUs.length === 0)
+})
+
+autoPickGPUsField.addEventListener('click', function() {
+    if (this.checked) {
+        $('#use_gpus').val([])
+    }
+
+    let gpuSettingEntry = getParameterSettingsEntry('use_gpus')
+    gpuSettingEntry.style.display = (this.checked ? 'none' : '')
+})
+
+async function getDiskPath() {
+    try {
+        var diskPath = getSetting("diskPath")
+        if (diskPath == '' || diskPath == undefined || diskPath == "undefined") {
+            let res = await fetch('/get/output_dir')
+            if (res.status === 200) {
+                res = await res.json()
+                res = res.output_dir
+
+                setSetting("diskPath", res)
+            }
+        }
+    } catch (e) {
+        console.log('error fetching output dir path', e)
+    }
+}
+
+async function getDevices() {
+    try {
+        let res = await fetch('/get/devices')
+        if (res.status === 200) {
+            res = await res.json()
+
+            let allDeviceIds = Object.keys(res['all']).filter(d => d !== 'cpu')
+            let activeDeviceIds = Object.keys(res['active']).filter(d => d !== 'cpu')
+
+            if (activeDeviceIds.length === 0) {
+                useCPUField.checked = true
+            }
+
+            if (allDeviceIds.length < MIN_GPUS_TO_SHOW_SELECTION || useCPUField.checked) {
+                let gpuSettingEntry = getParameterSettingsEntry('use_gpus')
+                gpuSettingEntry.style.display = 'none'
+                let autoPickGPUSettingEntry = getParameterSettingsEntry('auto_pick_gpus')
+                autoPickGPUSettingEntry.style.display = 'none'
+            }
+
+            if (allDeviceIds.length === 0) {
+                useCPUField.checked = true
+                useCPUField.disabled = true // no compatible GPUs, so make the CPU mandatory
+            }
+
+            autoPickGPUsField.checked = (res['config'] === 'auto')
+
+            useGPUsField.innerHTML = ''
+            allDeviceIds.forEach(device => {
+                let deviceName = res['all'][device]['name']
+                let deviceOption = `<option value="${device}">${deviceName} (${device})</option>`
+                useGPUsField.insertAdjacentHTML('beforeend', deviceOption)
+            })
+
+            if (autoPickGPUsField.checked) {
+                let gpuSettingEntry = getParameterSettingsEntry('use_gpus')
+                gpuSettingEntry.style.display = 'none'
+            } else {
+                $('#use_gpus').val(activeDeviceIds)
+            }
+        }
+    } catch (e) {
+        console.log('error fetching devices', e)
+    }
+}
+
+saveSettingsBtn.addEventListener('click', function() {
+	let updateBranch = (useBetaChannelField.checked ? 'beta' : 'main')
+
+	changeAppConfig({
+        'render_devices': getCurrentRenderDeviceSelection(),
+		'update_branch': updateBranch,
+		'ui_open_browser_on_start': uiOpenBrowserOnStartField.checked
+    })
+})

ui/media/js/plugins.js

@@ -0,0 +1,47 @@
+const PLUGIN_API_VERSION = "1.0"
+
+const PLUGINS = {
+    /**
+     * Register new buttons to show on each output image.
+     * 
+     * Example:
+     * PLUGINS['IMAGE_INFO_BUTTONS'].push({
+     *   text: 'Make a Similar Image',
+     *   on_click: function(origRequest, image) {
+     *     let newTaskRequest = getCurrentUserRequest()
+     *     newTaskRequest.reqBody = Object.assign({}, origRequest, {
+     *       init_image: image.src,
+     *       prompt_strength: 0.7,
+     *       seed: Math.floor(Math.random() * 10000000)
+     *     })
+     *     newTaskRequest.seed = newTaskRequest.reqBody.seed
+     *     createTask(newTaskRequest)
+     *   },
+     *   filter: function(origRequest, image) {
+     *     // this is an optional function. return true/false to show/hide the button
+     *     // if this function isn't set, the button will always be visible
+     *     return true
+     *   }
+     * })
+     */
+    IMAGE_INFO_BUTTONS: []
+}
+
+async function loadUIPlugins() {
+    try {
+        let res = await fetch('/get/ui_plugins')
+        if (res.status === 200) {
+            res = await res.json()
+            res.forEach(pluginPath => {
+                let script = document.createElement('script')
+                script.src = pluginPath + '?t=' + Date.now()
+
+                console.log('loading plugin', pluginPath)
+
+                document.head.appendChild(script)
+            })
+        }
+    } catch (e) {
+        console.log('error fetching plugin paths', e)
+    }
+}

ui/media/js/themes.js

@@ -0,0 +1,73 @@
+const themeField = document.getElementById("theme");
+var DEFAULT_THEME = {};
+var THEMES = []; // initialized in initTheme from data in css
+
+function getThemeName(theme) {
+    theme = theme.replace("theme-", "");
+    theme = theme.split("-").map(word => word.charAt(0).toUpperCase() + word.slice(1)).join(" ");
+    return theme;
+}
+// init themefield
+function initTheme() {
+    Array.from(document.styleSheets)
+        .filter(sheet => sheet.href?.startsWith(window.location.origin))
+        .flatMap(sheet => Array.from(sheet.cssRules))
+        .forEach(rule => {
+            var selector = rule.selectorText; // TODO: also do selector == ":root", re-run un-set props
+            if (selector && selector.startsWith(".theme-")) {
+                var theme_key = selector.substring(1);
+                THEMES.push({
+                    key: theme_key,
+                    name: getThemeName(theme_key),
+                    rule: rule
+                })
+            }
+            if (selector && selector == ":root") {
+                DEFAULT_THEME = {
+                    key: "theme-default",
+                    name: "Default",
+                    rule: rule
+                };
+            }
+    });
+    
+    THEMES.forEach(theme => {
+        var new_option = document.createElement("option");
+        new_option.setAttribute("value", theme.key);
+        new_option.innerText = theme.name;
+        themeField.appendChild(new_option);
+    });
+
+    
+    // setup the style transitions a second after app initializes, so initial style is instant
+    setTimeout(() => {
+        var body = document.querySelector("body");
+        var style = document.createElement('style');
+        style.innerHTML = "* { transition: background 0.5s, color 0.5s, background-color 0.5s; }";
+        body.appendChild(style);
+    }, 1000);
+}
+initTheme();
+
+function themeFieldChanged() {
+    var theme_key = themeField.value;
+
+    var body = document.querySelector("body");
+    body.classList.remove(...THEMES.map(theme => theme.key));
+    body.classList.add(theme_key);
+    
+    // 
+
+    body.style = "";
+    var theme = THEMES.find(t => t.key == theme_key);
+    if (theme) {
+        // refresh variables incase they are back referencing
+        Array.from(DEFAULT_THEME.rule.style)
+            .filter(cssVariable => !Array.from(theme.rule.style).includes(cssVariable))
+            .forEach(cssVariable => {
+            body.style.setProperty(cssVariable, DEFAULT_THEME.rule.style.getPropertyValue(cssVariable));
+        });
+    }
+}
+
+themeField.addEventListener('change', themeFieldChanged);

ui/media/js/utils.js

@@ -0,0 +1,360 @@
+// https://gomakethings.com/finding-the-next-and-previous-sibling-elements-that-match-a-selector-with-vanilla-js/
+function getNextSibling(elem, selector) {
+	// Get the next sibling element
+	var sibling = elem.nextElementSibling
+
+	// If there's no selector, return the first sibling
+	if (!selector) return sibling
+
+	// If the sibling matches our selector, use it
+	// If not, jump to the next sibling and continue the loop
+	while (sibling) {
+		if (sibling.matches(selector)) return sibling
+		sibling = sibling.nextElementSibling
+	}
+}
+
+
+
+/* Panel Stuff */
+
+// true = open
+var COLLAPSIBLES_INITIALIZED = false;
+const COLLAPSIBLES_KEY = "collapsibles";
+const COLLAPSIBLE_PANELS = []; // filled in by createCollapsibles with all the elements matching .collapsible
+
+// on-init call this for any panels that are marked open
+function toggleCollapsible(element) {
+    var collapsibleHeader = element.querySelector(".collapsible");
+    var handle = element.querySelector(".collapsible-handle");
+    collapsibleHeader.classList.toggle("active")
+    let content = getNextSibling(collapsibleHeader, '.collapsible-content')
+    if (!collapsibleHeader.classList.contains("active")) {
+        content.style.display = "none"
+        if (handle != null) {  // render results don't have a handle
+            handle.innerHTML = '➕' // plus
+        }
+    } else {
+        content.style.display = "block"
+        if (handle != null) {  // render results don't have a handle
+            handle.innerHTML = '➖' // minus
+        }
+    }
+    
+    if (COLLAPSIBLES_INITIALIZED && COLLAPSIBLE_PANELS.includes(element)) {
+        saveCollapsibles()
+    }
+}
+
+function saveCollapsibles() {
+    var values = {}
+    COLLAPSIBLE_PANELS.forEach(element => {
+        var value = element.querySelector(".collapsible").className.indexOf("active") !== -1
+        values[element.id] = value
+    })
+    localStorage.setItem(COLLAPSIBLES_KEY, JSON.stringify(values))
+}
+
+function createCollapsibles(node) {
+    var save = false
+    if (!node) {
+        node = document
+        save = true
+    }
+    let collapsibles = node.querySelectorAll(".collapsible")
+    collapsibles.forEach(function(c) {
+        if (save && c.parentElement.id) {
+            COLLAPSIBLE_PANELS.push(c.parentElement)
+        }
+        let handle = document.createElement('span')
+        handle.className = 'collapsible-handle'
+
+        if (c.classList.contains("active")) {
+            handle.innerHTML = '➖' // minus
+        } else {
+            handle.innerHTML = '➕' // plus
+        }
+        c.insertBefore(handle, c.firstChild)
+
+        c.addEventListener('click', function() {
+            toggleCollapsible(c.parentElement)
+        })
+    })
+    if (save) {
+        var saved = localStorage.getItem(COLLAPSIBLES_KEY)
+        if (!saved) { 
+            saved = tryLoadOldCollapsibles();
+        }
+        if (!saved) {
+            saveCollapsibles()
+            saved = localStorage.getItem(COLLAPSIBLES_KEY)
+        }
+        var values = JSON.parse(saved)
+        COLLAPSIBLE_PANELS.forEach(element => {
+            var value = element.querySelector(".collapsible").className.indexOf("active") !== -1
+            if (values[element.id] != value) {
+                toggleCollapsible(element)
+            }
+        })
+        COLLAPSIBLES_INITIALIZED = true
+    }
+}
+
+function tryLoadOldCollapsibles() {
+    var old_map = {
+        "advancedPanelOpen": "editor-settings",
+        "modifiersPanelOpen": "editor-modifiers",
+        "negativePromptPanelOpen": "editor-inputs-prompt"
+    };
+    if (localStorage.getItem(Object.keys(old_map)[0])) {
+        var result = {};
+        Object.keys(old_map).forEach(key => {
+            var value = localStorage.getItem(key);
+            if (value !== null) {
+                result[old_map[key]] = value == true || value == "true"
+                localStorage.removeItem(key)
+            }
+        });
+        result = JSON.stringify(result)
+        localStorage.setItem(COLLAPSIBLES_KEY, result)
+        return result
+    }
+    return null;
+}
+
+function permute(arr) {
+    let permutations = []
+    let n = arr.length
+    let n_permutations = Math.pow(2, n)
+    for (let i = 0; i < n_permutations; i++) {
+        let perm = []
+        let mask = Number(i).toString(2).padStart(n, '0')
+
+        for (let idx = 0; idx < mask.length; idx++) {
+            if (mask[idx] === '1' && arr[idx].trim() !== '') {
+                perm.push(arr[idx])
+            }
+        }
+
+        if (perm.length > 0) {
+            permutations.push(perm)
+        }
+    }
+
+    return permutations
+}
+
+// https://stackoverflow.com/a/8212878
+function millisecondsToStr(milliseconds) {
+    function numberEnding (number) {
+        return (number > 1) ? 's' : ''
+    }
+
+    var temp = Math.floor(milliseconds / 1000)
+    var hours = Math.floor((temp %= 86400) / 3600)
+    var s = ''
+    if (hours) {
+        s += hours + ' hour' + numberEnding(hours) + ' '
+    }
+    var minutes = Math.floor((temp %= 3600) / 60)
+    if (minutes) {
+        s += minutes + ' minute' + numberEnding(minutes) + ' '
+    }
+    var seconds = temp % 60
+    if (!hours && minutes < 4 && seconds) {
+        s += seconds + ' second' + numberEnding(seconds)
+    }
+
+    return s
+}
+
+// https://rosettacode.org/wiki/Brace_expansion#JavaScript
+function BraceExpander() {
+    'use strict'
+
+    // Index of any closing brace matching the opening
+    // brace at iPosn,
+    // with the indices of any immediately-enclosed commas.
+    function bracePair(tkns, iPosn, iNest, lstCommas) {
+        if (iPosn >= tkns.length || iPosn < 0) return null;
+
+        var t = tkns[iPosn],
+            n = (t === '{') ? (
+                iNest + 1
+            ) : (t === '}' ? (
+                iNest - 1
+            ) : iNest),
+            lst = (t === ',' && iNest === 1) ? (
+                lstCommas.concat(iPosn)
+            ) : lstCommas;
+
+        return n ? bracePair(tkns, iPosn + 1, n, lst) : {
+            close: iPosn,
+            commas: lst
+        };
+    }
+
+    // Parse of a SYNTAGM subtree
+    function andTree(dctSofar, tkns) {
+        if (!tkns.length) return [dctSofar, []];
+
+        var dctParse = dctSofar ? dctSofar : {
+                fn: and,
+                args: []
+            },
+
+            head = tkns[0],
+            tail = head ? tkns.slice(1) : [],
+
+            dctBrace = head === '{' ? bracePair(
+                tkns, 0, 0, []
+            ) : null,
+
+            lstOR = dctBrace && (
+                dctBrace.close
+            ) && dctBrace.commas.length ? (
+                splitAt(dctBrace.close + 1, tkns)
+            ) : null;
+
+        return andTree({
+            fn: and,
+            args: dctParse.args.concat(
+                lstOR ? (
+                    orTree(dctParse, lstOR[0], dctBrace.commas)
+                ) : head
+            )
+        }, lstOR ? (
+            lstOR[1]
+        ) : tail);
+    }
+
+    // Parse of a PARADIGM subtree
+    function orTree(dctSofar, tkns, lstCommas) {
+        if (!tkns.length) return [dctSofar, []];
+        var iLast = lstCommas.length;
+
+        return {
+            fn: or,
+            args: splitsAt(
+                lstCommas, tkns
+            ).map(function (x, i) {
+                var ts = x.slice(
+                    1, i === iLast ? (
+                        -1
+                    ) : void 0
+                );
+
+                return ts.length ? ts : [''];
+            }).map(function (ts) {
+                return ts.length > 1 ? (
+                    andTree(null, ts)[0]
+                ) : ts[0];
+            })
+        };
+    }
+
+    // List of unescaped braces and commas, and remaining strings
+    function tokens(str) {
+        // Filter function excludes empty splitting artefacts
+        var toS = function (x) {
+            return x.toString();
+        };
+
+        return str.split(/(\\\\)/).filter(toS).reduce(function (a, s) {
+            return a.concat(s.charAt(0) === '\\' ? s : s.split(
+                /(\\*[{,}])/
+            ).filter(toS));
+        }, []);
+    }
+
+    // PARSE TREE OPERATOR (1 of 2)
+    // Each possible head * each possible tail
+    function and(args) {
+        var lng = args.length,
+            head = lng ? args[0] : null,
+            lstHead = "string" === typeof head ? (
+                [head]
+            ) : head;
+
+        return lng ? (
+            1 < lng ? lstHead.reduce(function (a, h) {
+                return a.concat(
+                    and(args.slice(1)).map(function (t) {
+                        return h + t;
+                    })
+                );
+            }, []) : lstHead
+        ) : [];
+    }
+
+    // PARSE TREE OPERATOR (2 of 2)
+    // Each option flattened
+    function or(args) {
+        return args.reduce(function (a, b) {
+            return a.concat(b);
+        }, []);
+    }
+
+    // One list split into two (first sublist length n)
+    function splitAt(n, lst) {
+        return n < lst.length + 1 ? [
+            lst.slice(0, n), lst.slice(n)
+        ] : [lst, []];
+    }
+
+    // One list split into several (sublist lengths [n])
+    function splitsAt(lstN, lst) {
+        return lstN.reduceRight(function (a, x) {
+            return splitAt(x, a[0]).concat(a.slice(1));
+        }, [lst]);
+    }
+
+    // Value of the parse tree
+    function evaluated(e) {
+        return typeof e === 'string' ? e :
+            e.fn(e.args.map(evaluated));
+    }
+
+    // JSON prettyprint (for parse tree, token list etc)
+    function pp(e) {
+        return JSON.stringify(e, function (k, v) {
+            return typeof v === 'function' ? (
+                '[function ' + v.name + ']'
+            ) : v;
+        }, 2)
+    }
+
+
+    // ----------------------- MAIN ------------------------
+
+    // s -> [s]
+    this.expand = function(s) {
+        // BRACE EXPRESSION PARSED
+        var dctParse = andTree(null, tokens(s))[0];
+
+        // ABSTRACT SYNTAX TREE LOGGED
+        // console.log(pp(dctParse));
+
+        // AST EVALUATED TO LIST OF STRINGS
+        return evaluated(dctParse);
+    }
+
+}
+
+function asyncDelay(timeout) {
+    return new Promise(function(resolve, reject) {
+        setTimeout(resolve, timeout, true)
+    })
+}
+
+function preventNonNumericalInput(e) {
+    e = e || window.event;
+    let charCode = (typeof e.which == "undefined") ? e.keyCode : e.which;
+    let charStr = String.fromCharCode(charCode);
+    let re = e.target.getAttribute('pattern') || '^[0-9]+$'
+    re = new RegExp(re)
+
+    if (!charStr.match(re)) {
+        e.preventDefault();
+    }
+}

ui/media/main.css

@@ -1,413 +0,0 @@
-body {
-    font-family: Arial, Helvetica, sans-serif;
-    font-size: 11pt;
-    background-color: rgb(32, 33, 36);
-    color: #eee;
-}
-a {
-    color: rgb(0, 102, 204);
-}
-a:visited {
-    color: rgb(0, 102, 204);
-}
-label {
-    font-size: 10pt;
-}
-#prompt {
-    width: 100%;
-    height: 65pt;
-    box-sizing: border-box;
-}
-@media screen and (max-width: 600px) {
-    #prompt {
-        width: 95%;
-    }
-}
-.image_preview_container {
-    /* display: none; */
-    margin-top: 10pt;
-}
-.image_clear_btn {
-    position: absolute;
-    transform: translateX(-50%) translateY(-35%);
-    background: black;
-    color: white;
-    border: 2pt solid #ccc;
-    padding: 0;
-    cursor: pointer;
-    outline: inherit;
-    border-radius: 8pt;
-    width: 16pt;
-    height: 16pt;
-    font-family: Verdana;
-    font-size: 8pt;
-}
-.settings-box ul {
-    font-size: 9pt;
-    margin-bottom: 5px;
-    padding-left: 10px;
-    list-style-type: none;
-}
-.settings-box li {
-    padding-bottom: 4pt;
-}
-.editor-slider {
-    vertical-align: middle;
-}
-.outputMsg {
-    font-size: small;
-    padding-bottom: 3pt;
-}
-#progressBar {
-    font-size: small;
-}
-#footer {
-    font-size: small;
-    padding-left: 10pt;
-    background: none;
-}
-#footer-legal {
-    font-size: 8pt;
-}
-.imgSeedLabel {
-    font-size: 0.8em;
-    background-color: rgb(44, 45, 48);
-    border-radius: 3px;
-    padding: 5px;
-}
-.imgItem {
-    display: inline-block;
-    margin-top: 1em;
-    margin-right: 1em;
-}
-.imgContainer {
-    display: flex;
-    justify-content: flex-end;
-}
-.imgItemInfo {
-    padding-bottom: 0.5em;
-    display: flex;
-    align-items: flex-end;
-    flex-direction: column;
-    position: absolute;
-    padding: 5px;
-    opacity: 0;
-    transition: 0.1s all;
-}
-.imgContainer:hover > .imgItemInfo {
-    opacity: 1;
-}
-.imgItemInfo * {
-    margin-bottom: 7px;
-}
-#container {
-    width: 90%;
-    margin-left: auto;
-    margin-right: auto;
-}
-@media screen and (max-width: 1800px) {
-    #container {
-        width: 100%;
-    }
-}
-#logo small {
-    font-size: 11pt;
-}
-#editor {
-    padding: 5px;
-}
-#editor label {
-    font-weight: normal;
-}
-.settings-box label small {
-    color: rgb(153, 153, 153);
-}
-#preview {
-    padding: 5px;
-}
-#editor-inputs {
-    margin-bottom: 20px;
-}
-#editor-inputs-prompt {
-    flex: 1;
-}
-#editor-inputs .row {
-    padding-bottom: 10px;
-}
-#makeImage {
-    border-radius: 6px;
-}
-#editor-modifiers h5 {
-    padding: 5pt 0;
-    margin: 0;
-}
-#makeImage {
-    flex: 0 0 70px;
-    background: rgb(80, 0, 185);
-    border: 2px solid rgb(40, 0, 78);
-    color: rgb(255, 221, 255);
-    width: 100%;
-    height: 30pt;
-}
-#makeImage:hover {
-    background: rgb(93, 0, 214);
-}
-#stopImage {
-    flex: 0 0 70px;
-    background: rgb(132, 8, 0);
-    border: 2px solid rgb(122, 29, 0);
-    color: rgb(255, 221, 255);
-    width: 100%;
-    height: 30pt;
-    border-radius: 6px;
-    display: none;
-}
-#stopImage:hover {
-    background: rgb(177, 27, 0);
-}
-.flex-container {
-    display: flex;
-}
-.col-50 {
-    flex: 50%;
-}
-.col-fixed-10 {
-    flex: 0 0 380pt;
-}
-.col-free {
-    flex: 1;
-}
-.collapsible {
-    cursor: pointer;
-}
-.collapsible-content {
-    display: none;
-    padding-left: 15px;
-}
-.collapsible-content h5 {
-    padding: 5pt 0pt;
-    margin: 0;
-    font-size: 10pt;
-}
-.collapsible-handle {
-    color: white;
-    padding-right: 5px;
-}
-.panel-box {
-    background: rgb(44, 45, 48);
-    border: 1px solid rgb(47, 49, 53);
-    border-radius: 7px;
-    padding: 5px;
-    margin-bottom: 15px;
-    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
-}
-.panel-box h4 {
-    margin: 0;
-    padding: 2px 0;
-}
-#editor-modifiers .editor-modifiers-leaf {
-    padding-top: 10pt;
-    padding-bottom: 10pt;
-}
-#preview {
-    margin-left: 10pt;
-}
-img {
-    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
-}
-.line-separator {
-    background: rgb(56, 56, 56);
-    height: 1pt;
-    margin: 15pt 0;
-}
-#editor-inputs-tags-container {
-    margin-top: 5pt;
-    display: none;
-}
-#server-status {
-    display: inline;
-    float: right;
-    transform: translateY(-5pt);
-}
-#server-status-color {
-    /* width: 8pt;
-    height: 8pt;
-    border-radius: 4pt; */
-    font-size: 14pt;
-    color: rgb(128, 87, 0);
-    /* background-color: rgb(197, 1, 1); */
-    /* transform: translateY(15%); */
-    display: inline;
-}
-#server-status-msg {
-    color: rgb(128, 87, 0);
-    padding-left: 2pt;
-    font-size: 10pt;
-}
-.preview-prompt {
-    font-size: 16pt;
-    margin-bottom: 10pt;
-}
-#coffeeButton {
-    height: 23px;
-    transform: translateY(25%);
-}
-
-#inpaintingEditor {
-    width: 300pt;
-    height: 300pt;
-    margin-top: 5pt;
-}
-.drawing-board-canvas-wrapper {
-    background-size: 100% 100%;
-}
-.drawing-board-control > button {
-    background-color: #eee;
-    border-radius: 3pt;
-}
-.drawing-board-control-inner {
-    background-color: #eee;
-    border-radius: 3pt;
-}
-#inpaintingEditor canvas {
-    opacity: 0.6;
-}
-#enable_mask {
-    margin-top: 8pt;
-}
-
-#top-nav {
-    padding-top: 3pt;
-    padding-bottom: 15pt;
-}
-#top-nav .icon {
-    padding-right: 4pt;
-    font-size: 14pt;
-    transform: translateY(1pt);
-}
-#logo {
-    display: inline;
-}
-#logo h1 {
-    display: inline;
-}
-#top-nav-items {
-    list-style-type: none;
-    display: inline;
-    float: right;
-}
-#top-nav-items > li {
-    float: left;
-    display: inline;
-    padding-left: 20pt;
-    cursor: default;
-}
-#initial-text {
-    padding-top: 15pt;
-    padding-left: 4pt;
-}
-.settings-subheader {
-    font-size: 10pt;
-    font-weight: bold;
-}
-.pl-5 {
-    padding-left: 5pt;
-}
-#system-settings {
-    width: 360pt;
-    transform: translateX(-100%) translateX(70pt);
-
-    padding-top: 10pt;
-    padding-bottom: 10pt;
-}
-#system-settings ul {
-    margin: 0;
-    padding: 0;
-}
-#system-settings li {
-    padding-left: 5pt;
-}
-#community-links {
-    list-style-type: none;
-    margin: 0;
-    padding: 12pt;
-    padding-bottom: 0pt;
-    transform: translateX(-15%);
-}
-#community-links li {
-    padding-bottom: 12pt;
-    display: block;
-    font-size: 10pt;
-}
-#community-links li .fa-fw {
-    padding-right: 2pt;
-}
-#community-links li a {
-    color: white;
-    text-decoration: none;
-}
-.dropdown {
-    overflow: hidden;
-}
-.dropdown-content {
-    display: none;
-    position: absolute;
-    z-index: 2;
-
-    background: rgb(18, 18, 19);
-    border: 2px solid rgb(37, 38, 41);
-    border-radius: 7px;
-    padding: 5px;
-    margin-bottom: 15px;
-    box-shadow: 0 20px 28px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
-}
-.dropdown:hover .dropdown-content {
-    display: block;
-}
-
-.imageTaskContainer {
-    border: 1px solid #333;
-    margin-bottom: 10pt;
-    padding: 5pt;
-    border-radius: 5pt;
-    box-shadow: 0 20px 28px 0 rgba(0, 0, 0, 0.15), 0 6px 20px 0 rgba(0, 0, 0, 0.15);
-}
-.taskStatusLabel {
-    float: left;
-    font-size: 8pt;
-    background:rgb(44, 45, 48);
-    border: 1px solid rgb(61, 62, 66);
-    padding: 2pt 4pt;
-    border-radius: 2pt;
-    margin-right: 5pt;
-}
-.activeTaskLabel {
-    background:rgb(0, 90, 30);
-    border: 1px solid rgb(0, 75, 19);
-    color:rgb(204, 255, 217)
-}
-.secondaryButton {
-    background: rgb(132, 8, 0);
-    border: 1px solid rgb(122, 29, 0);
-    color: rgb(255, 221, 255);
-    padding: 3pt 6pt;
-    border-radius: 5px;
-}
-.secondaryButton:hover {
-    background: rgb(177, 27, 0);
-}
-.stopTask {
-    float: right;
-}
-#preview-tools {
-    display: none;
-    padding: 4pt;
-}
-.taskConfig {
-    font-size: 10pt;
-    color: #aaa;
-    margin-bottom: 5pt;
-}
-.img-batch {
-    display: inline;
-}

ui/sd_internal/__init__.py

@@ -18,11 +18,11 @@ class Request:
     precision: str = "autocast" # or "full"
     save_to_disk_path: str = None
     turbo: bool = True
-    use_cpu: bool = False
     use_full_precision: bool = False
     use_face_correction: str = None # or "GFPGANv1.3"
     use_upscale: str = None # or "RealESRGAN_x4plus" or "RealESRGAN_x4plus_anime_6B"
     use_stable_diffusion_model: str = "sd-v1-4"
+    use_vae_model: str = None
     show_only_filtered_image: bool = False
     output_format: str = "jpeg" # or "png"
 
@@ -45,10 +45,11 @@ class Request:
             "use_face_correction": self.use_face_correction,
             "use_upscale": self.use_upscale,
             "use_stable_diffusion_model": self.use_stable_diffusion_model,
+            "use_vae_model": self.use_vae_model,
             "output_format": self.output_format,
         }
 
-    def to_string(self):
+    def __str__(self):
         return f'''
     session_id: {self.session_id}
     prompt: {self.prompt}
@@ -62,11 +63,11 @@ class Request:
     precision: {self.precision}
     save_to_disk_path: {self.save_to_disk_path}
     turbo: {self.turbo}
-    use_cpu: {self.use_cpu}
     use_full_precision: {self.use_full_precision}
     use_face_correction: {self.use_face_correction}
     use_upscale: {self.use_upscale}
     use_stable_diffusion_model: {self.use_stable_diffusion_model}
+    use_vae_model: {self.use_vae_model}
     show_only_filtered_image: {self.show_only_filtered_image}
     output_format: {self.output_format}
 

ui/sd_internal/device_manager.py

@@ -0,0 +1,168 @@
+import os
+import torch
+import traceback
+import re
+
+COMPARABLE_GPU_PERCENTILE = 0.65 # if a GPU's free_mem is within this % of the GPU with the most free_mem, it will be picked
+
+mem_free_threshold = 0
+
+def get_device_delta(render_devices, active_devices):
+    '''
+    render_devices: 'cpu', or 'auto' or ['cuda:N'...]
+    active_devices: ['cpu', 'cuda:N'...]
+    '''
+
+    if render_devices in ('cpu', 'auto'):
+        render_devices = [render_devices]
+    elif render_devices is not None:
+        if isinstance(render_devices, str):
+            render_devices = [render_devices]
+        if isinstance(render_devices, list) and len(render_devices) > 0:
+            render_devices = list(filter(lambda x: x.startswith('cuda:'), render_devices))
+            if len(render_devices) == 0:
+                raise Exception('Invalid render_devices value in config.json. Valid: {"render_devices": ["cuda:0", "cuda:1"...]}, or {"render_devices": "cpu"} or {"render_devices": "auto"}')
+
+            render_devices = list(filter(lambda x: is_device_compatible(x), render_devices))
+            if len(render_devices) == 0:
+                raise Exception('Sorry, none of the render_devices configured in config.json are compatible with Stable Diffusion')
+        else:
+            raise Exception('Invalid render_devices value in config.json. Valid: {"render_devices": ["cuda:0", "cuda:1"...]}, or {"render_devices": "cpu"} or {"render_devices": "auto"}')
+    else:
+        render_devices = ['auto']
+
+    if 'auto' in render_devices:
+        render_devices = auto_pick_devices(active_devices)
+        if 'cpu' in render_devices:
+            print('WARNING: Could not find a compatible GPU. Using the CPU, but this will be very slow!')
+
+    active_devices = set(active_devices)
+    render_devices = set(render_devices)
+
+    devices_to_start = render_devices - active_devices
+    devices_to_stop = active_devices - render_devices
+
+    return devices_to_start, devices_to_stop
+
+def auto_pick_devices(currently_active_devices):
+    global mem_free_threshold
+
+    if not torch.cuda.is_available(): return ['cpu']
+
+    device_count = torch.cuda.device_count()
+    if device_count == 1:
+        return ['cuda:0'] if is_device_compatible('cuda:0') else ['cpu']
+
+    print('Autoselecting GPU. Using most free memory.')
+    devices = []
+    for device in range(device_count):
+        device = f'cuda:{device}'
+        if not is_device_compatible(device):
+            continue
+
+        mem_free, mem_total = torch.cuda.mem_get_info(device)
+        mem_free /= float(10**9)
+        mem_total /= float(10**9)
+        device_name = torch.cuda.get_device_name(device)
+        print(f'{device} detected: {device_name} - Memory (free/total): {round(mem_free, 2)}Gb / {round(mem_total, 2)}Gb')
+        devices.append({'device': device, 'device_name': device_name, 'mem_free': mem_free})
+
+    devices.sort(key=lambda x:x['mem_free'], reverse=True)
+    max_mem_free = devices[0]['mem_free']
+    curr_mem_free_threshold = COMPARABLE_GPU_PERCENTILE * max_mem_free
+    mem_free_threshold = max(curr_mem_free_threshold, mem_free_threshold)
+
+    # Auto-pick algorithm:
+    # 1. Pick the top 75 percentile of the GPUs, sorted by free_mem.
+    # 2. Also include already-running devices (GPU-only), otherwise their free_mem will
+    #    always be very low (since their VRAM contains the model).
+    #    These already-running devices probably aren't terrible, since they were picked in the past.
+    #    Worst case, the user can restart the program and that'll get rid of them.
+    devices = list(filter((lambda x: x['mem_free'] > mem_free_threshold or x['device'] in currently_active_devices), devices))
+    devices = list(map(lambda x: x['device'], devices))
+    return devices
+
+def device_init(thread_data, device):
+    '''
+    This function assumes the 'device' has already been verified to be compatible.
+    `get_device_delta()` has already filtered out incompatible devices.
+    '''
+
+    validate_device_id(device, log_prefix='device_init')
+
+    if device == 'cpu':
+        thread_data.device = 'cpu'
+        thread_data.device_name = get_processor_name()
+        print('Render device CPU available as', thread_data.device_name)
+        return
+
+    thread_data.device_name = torch.cuda.get_device_name(device)
+    thread_data.device = device
+
+    # Force full precision on 1660 and 1650 NVIDIA cards to avoid creating green images
+    device_name = thread_data.device_name.lower()
+    thread_data.force_full_precision = ('nvidia' in device_name or 'geforce' in device_name) and (' 1660' in device_name or ' 1650' in device_name)
+    if thread_data.force_full_precision:
+        print('forcing full precision on NVIDIA 16xx cards, to avoid green images. GPU detected: ', thread_data.device_name)
+        # Apply force_full_precision now before models are loaded.
+        thread_data.precision = 'full'
+
+    print(f'Setting {device} as active')
+    torch.cuda.device(device)
+
+    return
+
+def validate_device_id(device, log_prefix=''):
+    def is_valid():
+        if not isinstance(device, str):
+            return False
+        if device == 'cpu':
+            return True
+        if not device.startswith('cuda:') or not device[5:].isnumeric():
+            return False
+        return True
+
+    if not is_valid():
+        raise EnvironmentError(f"{log_prefix}: device id should be 'cpu', or 'cuda:N' (where N is an integer index for the GPU). Got: {device}")
+
+def is_device_compatible(device):
+    '''
+    Returns True/False, and prints any compatibility errors
+    '''
+    try:
+        validate_device_id(device, log_prefix='is_device_compatible')
+    except:
+        print(str(e))
+        return False
+
+    if device == 'cpu': return True
+    # Memory check
+    try:
+        _, mem_total = torch.cuda.mem_get_info(device)
+        mem_total /= float(10**9)
+        if mem_total < 3.0:
+            print(f'GPU {device} with less than 3 GB of VRAM is not compatible with Stable Diffusion')
+            return False
+    except RuntimeError as e:
+        print(str(e))
+        return False
+    return True
+
+def get_processor_name():
+    try:
+        import platform, subprocess
+        if platform.system() == "Windows":
+            return platform.processor()
+        elif platform.system() == "Darwin":
+            os.environ['PATH'] = os.environ['PATH'] + os.pathsep + '/usr/sbin'
+            command = "sysctl -n machdep.cpu.brand_string"
+            return subprocess.check_output(command).strip()
+        elif platform.system() == "Linux":
+            command = "cat /proc/cpuinfo"
+            all_info = subprocess.check_output(command, shell=True).decode().strip()
+            for line in all_info.split("\n"):
+                if "model name" in line:
+                    return re.sub(".*model name.*:", "", line, 1).strip()
+    except:
+        print(traceback.format_exc())
+        return "cpu"

ui/sd_internal/runtime.py

@@ -1,8 +1,15 @@
+"""runtime.py: torch device owned by a thread.
+Notes:
+    Avoid device switching, transfering all models will get too complex.
+    To use a diffrent device signal the current render device to exit
+    And then start a new clean thread for the new device.
+"""
 import json
 import os, re
 import traceback
 import torch
 import numpy as np
+from gc import collect as gc_collect
 from omegaconf import OmegaConf
 from PIL import Image, ImageOps
 from tqdm import tqdm, trange
@@ -28,69 +35,64 @@ logging.set_verbosity_error()
 # consts
 config_yaml = "optimizedSD/v1-inference.yaml"
 filename_regex = re.compile('[^a-zA-Z0-9]')
+force_gfpgan_to_cuda0 = True # workaround: gfpgan currently works only on cuda:0
 
 # api stuff
+from sd_internal import device_manager
 from . import Request, Response, Image as ResponseImage
 import base64
 from io import BytesIO
 #from colorama import Fore
 
-# local
-stop_processing = False
-temp_images = {}
+from threading import local as LocalThreadVars
+thread_data = LocalThreadVars()
 
-ckpt_file = None
-gfpgan_file = None
-real_esrgan_file = None
+def thread_init(device):
+    # Thread bound properties
+    thread_data.stop_processing = False
+    thread_data.temp_images = {}
 
-model = None
-modelCS = None
-modelFS = None
-model_gfpgan = None
-model_real_esrgan = None
+    thread_data.ckpt_file = None
+    thread_data.vae_file = None
+    thread_data.gfpgan_file = None
+    thread_data.real_esrgan_file = None
 
-model_is_half = False
-model_fs_is_half = False
-device = None
-unet_bs = 1
-precision = 'autocast'
-sampler_plms = None
-sampler_ddim = None
+    thread_data.model = None
+    thread_data.modelCS = None
+    thread_data.modelFS = None
+    thread_data.model_gfpgan = None
+    thread_data.model_real_esrgan = None
 
-has_valid_gpu = False
-force_full_precision = False
-try:
-    gpu = torch.cuda.current_device()
-    gpu_name = torch.cuda.get_device_name(gpu)
-    print('GPU detected: ', gpu_name)
+    thread_data.model_is_half = False
+    thread_data.model_fs_is_half = False
+    thread_data.device = None
+    thread_data.device_name = None
+    thread_data.unet_bs = 1
+    thread_data.precision = 'autocast'
+    thread_data.sampler_plms = None
+    thread_data.sampler_ddim = None
 
-    force_full_precision = ('nvidia' in gpu_name.lower() or 'geforce' in gpu_name.lower()) and (' 1660' in gpu_name or ' 1650' in gpu_name) # otherwise these NVIDIA cards create green images
-    if force_full_precision:
-        print('forcing full precision on NVIDIA 16xx cards, to avoid green images. GPU detected: ', gpu_name)
+    thread_data.turbo = False
+    thread_data.force_full_precision = False
+    thread_data.reduced_memory = True
 
-    mem_free, mem_total = torch.cuda.mem_get_info(gpu)
-    mem_total /= float(10**9)
-    if mem_total < 3.0:
-        print("GPUs with less than 3 GB of VRAM are not compatible with Stable Diffusion")
-        raise Exception()
+    device_manager.device_init(thread_data, device)
 
-    has_valid_gpu = True
-except:
-    print('WARNING: No compatible GPU found. Using the CPU, but this will be very slow!')
-    pass
+def load_model_ckpt():
+    if not thread_data.ckpt_file: raise ValueError(f'Thread ckpt_file is undefined.')
+    if not os.path.exists(thread_data.ckpt_file + '.ckpt'): raise FileNotFoundError(f'Cannot find {thread_data.ckpt_file}.ckpt')
 
-def load_model_ckpt(ckpt_to_use, device_to_use='cuda', turbo=False, unet_bs_to_use=1, precision_to_use='autocast'):
-    global ckpt_file, model, modelCS, modelFS, model_is_half, device, unet_bs, precision, model_fs_is_half
+    if not thread_data.precision:
+        thread_data.precision = 'full' if thread_data.force_full_precision else 'autocast'
 
-    ckpt_file = ckpt_to_use
-    device = device_to_use if has_valid_gpu else 'cpu'
-    precision = precision_to_use if not force_full_precision else 'full'
-    unet_bs = unet_bs_to_use
+    if not thread_data.unet_bs:
+        thread_data.unet_bs = 1
 
-    if device == 'cpu':
-        precision = 'full'
+    if thread_data.device == 'cpu':
+        thread_data.precision = 'full'
 
-    sd = load_model_from_config(f"{ckpt_file}.ckpt")
+    print('loading', thread_data.ckpt_file + '.ckpt', 'to device', thread_data.device, 'using precision', thread_data.precision)
+    sd = load_model_from_config(thread_data.ckpt_file + '.ckpt')
     li, lo = [], []
     for key, value in sd.items():
         sp = key.split(".")
@@ -113,74 +115,208 @@ def load_model_ckpt(ckpt_to_use, device_to_use='cuda', turbo=False, unet_bs_to_u
     model = instantiate_from_config(config.modelUNet)
     _, _ = model.load_state_dict(sd, strict=False)
     model.eval()
-    model.cdevice = device
-    model.unet_bs = unet_bs
-    model.turbo = turbo
+    model.cdevice = torch.device(thread_data.device)
+    model.unet_bs = thread_data.unet_bs
+    model.turbo = thread_data.turbo
+    # if thread_data.device != 'cpu':
+    #     model.to(thread_data.device)
+    #if thread_data.reduced_memory:
+        #model.model1.to("cpu")
+        #model.model2.to("cpu")
+    thread_data.model = model
 
     modelCS = instantiate_from_config(config.modelCondStage)
     _, _ = modelCS.load_state_dict(sd, strict=False)
     modelCS.eval()
-    modelCS.cond_stage_model.device = device
+    modelCS.cond_stage_model.device = torch.device(thread_data.device)
+    # if thread_data.device != 'cpu':
+    #     if thread_data.reduced_memory:
+    #         modelCS.to('cpu')
+    #     else:
+    #         modelCS.to(thread_data.device) # Preload on device if not already there.
+    thread_data.modelCS = modelCS
 
     modelFS = instantiate_from_config(config.modelFirstStage)
     _, _ = modelFS.load_state_dict(sd, strict=False)
+
+    if thread_data.vae_file is not None:
+        for model_extension in ['.ckpt', '.vae.pt']:
+            if os.path.exists(thread_data.vae_file + model_extension):
+                print(f"Loading VAE weights from: {thread_data.vae_file}{model_extension}")
+                vae_ckpt = torch.load(thread_data.vae_file + model_extension, map_location="cpu")
+                vae_dict = {k: v for k, v in vae_ckpt["state_dict"].items() if k[0:4] != "loss"}
+                modelFS.first_stage_model.load_state_dict(vae_dict, strict=False)
+                break
+            else:
+                print(f'Cannot find VAE file: {thread_data.vae_file}{model_extension}')
+
     modelFS.eval()
+    # if thread_data.device != 'cpu':
+    #     if thread_data.reduced_memory:
+    #         modelFS.to('cpu')
+    #     else:
+    #         modelFS.to(thread_data.device) # Preload on device if not already there.
+    thread_data.modelFS = modelFS
     del sd
 
-    if device != "cpu" and precision == "autocast":
-        model.half()
-        modelCS.half()
-        modelFS.half()
-        model_is_half = True
-        model_fs_is_half = True
+    if thread_data.device != "cpu" and thread_data.precision == "autocast":
+        thread_data.model.half()
+        thread_data.modelCS.half()
+        thread_data.modelFS.half()
+        thread_data.model_is_half = True
+        thread_data.model_fs_is_half = True
     else:
-        model_is_half = False
-        model_fs_is_half = False
+        thread_data.model_is_half = False
+        thread_data.model_fs_is_half = False
 
-    print('loaded ', ckpt_file, 'to', device, 'precision', precision)
+    print(f'''loaded model
+ model file: {thread_data.ckpt_file}.ckpt
+ model.device: {model.device}
+ modelCS.device: {modelCS.cond_stage_model.device}
+ modelFS.device: {thread_data.modelFS.device}
+ using precision: {thread_data.precision}''')
 
-def load_model_gfpgan(gfpgan_to_use):
-    global gfpgan_file, model_gfpgan
+def unload_filters():
+    if thread_data.model_gfpgan is not None:
+        if thread_data.device != 'cpu': thread_data.model_gfpgan.gfpgan.to('cpu')
 
-    if gfpgan_to_use is None:
-        return
+        del thread_data.model_gfpgan
+    thread_data.model_gfpgan = None
 
-    gfpgan_file = gfpgan_to_use
-    model_path = gfpgan_to_use + ".pth"
+    if thread_data.model_real_esrgan is not None:
+        if thread_data.device != 'cpu': thread_data.model_real_esrgan.model.to('cpu')
 
-    if device == 'cpu':
-        model_gfpgan = GFPGANer(model_path=model_path, upscale=1, arch='clean', channel_multiplier=2, bg_upsampler=None, device=torch.device('cpu'))
-    else:
-        model_gfpgan = GFPGANer(model_path=model_path, upscale=1, arch='clean', channel_multiplier=2, bg_upsampler=None, device=torch.device('cuda'))
+        del thread_data.model_real_esrgan
+    thread_data.model_real_esrgan = None
 
-    print('loaded ', gfpgan_to_use, 'to', device, 'precision', precision)
+    gc()
 
-def load_model_real_esrgan(real_esrgan_to_use):
-    global real_esrgan_file, model_real_esrgan
+def unload_models():
+    if thread_data.model is not None:
+        print('Unloading models...')
+        if thread_data.device != 'cpu':
+            thread_data.modelFS.to('cpu')
+            thread_data.modelCS.to('cpu')
+            thread_data.model.model1.to("cpu")
+            thread_data.model.model2.to("cpu")
 
-    if real_esrgan_to_use is None:
-        return
+        del thread_data.model
+        del thread_data.modelCS
+        del thread_data.modelFS
 
-    real_esrgan_file = real_esrgan_to_use
-    model_path = real_esrgan_to_use + ".pth"
+    thread_data.model = None
+    thread_data.modelCS = None
+    thread_data.modelFS = None
+
+    gc()
+
+def wait_model_move_to(model, target_device): # Send to target_device and wait until complete.
+    if thread_data.device == target_device: return
+    start_mem = torch.cuda.memory_allocated(thread_data.device) / 1e6
+    if start_mem <= 0: return
+    model_name = model.__class__.__name__
+    print(f'Device {thread_data.device} - Sending model {model_name} to {target_device} | Memory transfer starting. Memory Used: {round(start_mem)}Mb')
+    start_time = time.time()
+    model.to(target_device)
+    time_step = start_time
+    WARNING_TIMEOUT = 1.5 # seconds - Show activity in console after timeout.
+    last_mem = start_mem
+    is_transfering = True
+    while is_transfering:
+        time.sleep(0.5) # 500ms
+        mem = torch.cuda.memory_allocated(thread_data.device) / 1e6
+        is_transfering = bool(mem > 0 and mem < last_mem) # still stuff loaded, but less than last time.
+        last_mem = mem
+        if not is_transfering:
+            break;
+        if time.time() - time_step > WARNING_TIMEOUT: # Long delay, print to console to show activity.
+            print(f'Device {thread_data.device} - Waiting for Memory transfer. Memory Used: {round(mem)}Mb, Transfered: {round(start_mem - mem)}Mb')
+            time_step = time.time()
+    print(f'Device {thread_data.device} - {model_name} Moved: {round(start_mem - last_mem)}Mb in {round(time.time() - start_time, 3)} seconds to {target_device}')
+
+def load_model_gfpgan():
+    if thread_data.gfpgan_file is None: raise ValueError(f'Thread gfpgan_file is undefined.')
+    model_path = thread_data.gfpgan_file + ".pth"
+    device = 'cuda:0' if force_gfpgan_to_cuda0 else thread_data.device
+    thread_data.model_gfpgan = GFPGANer(device=torch.device(device), model_path=model_path, upscale=1, arch='clean', channel_multiplier=2, bg_upsampler=None)
+    print('loaded', thread_data.gfpgan_file, 'to', thread_data.model_gfpgan.device, 'precision', thread_data.precision)
+
+def load_model_real_esrgan():
+    if thread_data.real_esrgan_file is None: raise ValueError(f'Thread real_esrgan_file is undefined.')
+    model_path = thread_data.real_esrgan_file + ".pth"
 
     RealESRGAN_models = {
         'RealESRGAN_x4plus': RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=4),
         'RealESRGAN_x4plus_anime_6B': RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=6, num_grow_ch=32, scale=4)
     }
 
-    model_to_use = RealESRGAN_models[real_esrgan_to_use]
+    model_to_use = RealESRGAN_models[thread_data.real_esrgan_file]
 
-    if device == 'cpu':
-        model_real_esrgan = RealESRGANer(scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=False) # cpu does not support half
-        model_real_esrgan.device = torch.device('cpu')
-        model_real_esrgan.model.to('cpu')
+    if thread_data.device == 'cpu':
+        thread_data.model_real_esrgan = RealESRGANer(device=torch.device(thread_data.device), scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=False) # cpu does not support half
+        #thread_data.model_real_esrgan.device = torch.device(thread_data.device)
+        thread_data.model_real_esrgan.model.to('cpu')
     else:
-        model_real_esrgan = RealESRGANer(scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=model_is_half)
+        thread_data.model_real_esrgan = RealESRGANer(device=torch.device(thread_data.device), scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=thread_data.model_is_half)
 
-    model_real_esrgan.model.name = real_esrgan_to_use
+    thread_data.model_real_esrgan.model.name = thread_data.real_esrgan_file
+    print('loaded ', thread_data.real_esrgan_file, 'to', thread_data.model_real_esrgan.device, 'precision', thread_data.precision)
 
-    print('loaded ', real_esrgan_to_use, 'to', device, 'precision', precision)
+
+def get_session_out_path(disk_path, session_id):
+    if disk_path is None: return None
+    if session_id is None: return None
+
+    session_out_path = os.path.join(disk_path, filename_regex.sub('_',session_id))
+    os.makedirs(session_out_path, exist_ok=True)
+    return session_out_path
+
+def get_base_path(disk_path, session_id, prompt, img_id, ext, suffix=None):
+    if disk_path is None: return None
+    if session_id is None: return None
+    if ext is None: raise Exception('Missing ext')
+
+    session_out_path = get_session_out_path(disk_path, session_id)
+
+    prompt_flattened = filename_regex.sub('_', prompt)[:50]
+
+    if suffix is not None:
+        return os.path.join(session_out_path, f"{prompt_flattened}_{img_id}_{suffix}.{ext}")
+    return os.path.join(session_out_path, f"{prompt_flattened}_{img_id}.{ext}")
+
+def apply_filters(filter_name, image_data, model_path=None):
+    print(f'Applying filter {filter_name}...')
+    gc() # Free space before loading new data.
+
+    if filter_name == 'gfpgan':
+        if isinstance(image_data, torch.Tensor):
+            image_data.to('cuda:0' if force_gfpgan_to_cuda0 else thread_data.device)
+
+        if model_path is not None and model_path != thread_data.gfpgan_file:
+            thread_data.gfpgan_file = model_path
+            load_model_gfpgan()
+        elif not thread_data.model_gfpgan:
+            load_model_gfpgan()
+        if thread_data.model_gfpgan is None: raise Exception('Model "gfpgan" not loaded.')
+        print('enhance with', thread_data.gfpgan_file, 'on', thread_data.model_gfpgan.device, 'precision', thread_data.precision)
+        _, _, output = thread_data.model_gfpgan.enhance(image_data[:,:,::-1], has_aligned=False, only_center_face=False, paste_back=True)
+        image_data = output[:,:,::-1]
+
+    if filter_name == 'real_esrgan':
+        if isinstance(image_data, torch.Tensor):
+            image_data.to(thread_data.device)
+
+        if model_path is not None and model_path != thread_data.real_esrgan_file:
+            thread_data.real_esrgan_file = model_path
+            load_model_real_esrgan()
+        elif not thread_data.model_real_esrgan:
+            load_model_real_esrgan()
+        if thread_data.model_real_esrgan is None: raise Exception('Model "gfpgan" not loaded.')
+        print('enhance with', thread_data.real_esrgan_file, 'on', thread_data.model_real_esrgan.device, 'precision', thread_data.precision)
+        output, _ = thread_data.model_real_esrgan.enhance(image_data[:,:,::-1])
+        image_data = output[:,:,::-1]
+
+    return image_data
 
 def mk_img(req: Request):
     try:
@@ -188,117 +324,121 @@ def mk_img(req: Request):
     except Exception as e:
         print(traceback.format_exc())
 
-        gc()
-
-        if device != "cpu":
-            modelFS.to("cpu")
-            modelCS.to("cpu")
-
-            model.model1.to("cpu")
-            model.model2.to("cpu")
-
-        gc()
+        if thread_data.device != 'cpu':
+            thread_data.modelFS.to('cpu')
+            thread_data.modelCS.to('cpu')
+            thread_data.model.model1.to("cpu")
+            thread_data.model.model2.to("cpu")
 
+        gc() # Release from memory.
         yield json.dumps({
             "status": 'failed',
             "detail": str(e)
         })
 
-def do_mk_img(req: Request):
-    global ckpt_file
-    global model, modelCS, modelFS, device
-    global model_gfpgan, model_real_esrgan
-    global stop_processing
+def update_temp_img(req, x_samples):
+    partial_images = []
+    for i in range(req.num_outputs):
+        x_sample_ddim = thread_data.modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
+        x_sample = torch.clamp((x_sample_ddim + 1.0) / 2.0, min=0.0, max=1.0)
+        x_sample = 255.0 * rearrange(x_sample[0].cpu().numpy(), "c h w -> h w c")
+        x_sample = x_sample.astype(np.uint8)
+        img = Image.fromarray(x_sample)
+        buf = BytesIO()
+        img.save(buf, format='JPEG')
+        buf.seek(0)
 
-    stop_processing = False
+        del img, x_sample, x_sample_ddim
+        # don't delete x_samples, it is used in the code that called this callback
+
+        thread_data.temp_images[str(req.session_id) + '/' + str(i)] = buf
+        partial_images.append({'path': f'/image/tmp/{req.session_id}/{i}'})
+    return partial_images
+
+# Build and return the apropriate generator for do_mk_img
+def get_image_progress_generator(req, extra_props=None):
+    if not req.stream_progress_updates:
+        def empty_callback(x_samples, i): return x_samples
+        return empty_callback
+
+    thread_data.partial_x_samples = None
+    last_callback_time = -1
+    def img_callback(x_samples, i):
+        nonlocal last_callback_time
+
+        thread_data.partial_x_samples = x_samples
+        step_time = time.time() - last_callback_time if last_callback_time != -1 else -1
+        last_callback_time = time.time()
+
+        progress = {"step": i, "step_time": step_time}
+        if extra_props is not None:
+            progress.update(extra_props)
+
+        if req.stream_image_progress and i % 5 == 0:
+            progress['output'] = update_temp_img(req, x_samples)
+
+        yield json.dumps(progress)
+
+        if thread_data.stop_processing:
+            raise UserInitiatedStop("User requested that we stop processing")
+    return img_callback
+
+def do_mk_img(req: Request):
+    thread_data.stop_processing = False
 
     res = Response()
     res.request = req
     res.images = []
 
-    temp_images.clear()
+    thread_data.temp_images.clear()
 
     # custom model support:
     #  the req.use_stable_diffusion_model needs to be a valid path
     #  to the ckpt file (without the extension).
+    if not os.path.exists(req.use_stable_diffusion_model + '.ckpt'): raise FileNotFoundError(f'Cannot find {req.use_stable_diffusion_model}.ckpt')
 
     needs_model_reload = False
-    if ckpt_file != req.use_stable_diffusion_model:
-        ckpt_file = req.use_stable_diffusion_model
+    if not thread_data.model or thread_data.ckpt_file != req.use_stable_diffusion_model or thread_data.vae_file != req.use_vae_model:
+        thread_data.ckpt_file = req.use_stable_diffusion_model
+        thread_data.vae_file = req.use_vae_model
         needs_model_reload = True
 
-    model.turbo = req.turbo
-    if req.use_cpu:
-        if device != 'cpu':
-            device = 'cpu'
-
-            if model_is_half:
-                del model, modelCS, modelFS
-                load_model_ckpt(ckpt_file, device)
-                needs_model_reload = False
-
-            load_model_gfpgan(gfpgan_file)
-            load_model_real_esrgan(real_esrgan_file)
-    else:
-        if has_valid_gpu:
-            prev_device = device
-            device = 'cuda'
-
-            if (precision == 'autocast' and (req.use_full_precision or not model_is_half)) or \
-                (precision == 'full' and not req.use_full_precision and not force_full_precision):
-
-                del model, modelCS, modelFS
-                load_model_ckpt(ckpt_file, device, req.turbo, unet_bs, ('full' if req.use_full_precision else 'autocast'))
-                needs_model_reload = False
-
-                if prev_device != device:
-                    load_model_gfpgan(gfpgan_file)
-                    load_model_real_esrgan(real_esrgan_file)
+    if thread_data.device != 'cpu':
+        if (thread_data.precision == 'autocast' and (req.use_full_precision or not thread_data.model_is_half)) or \
+            (thread_data.precision == 'full' and not req.use_full_precision and not thread_data.force_full_precision):
+            thread_data.precision = 'full' if req.use_full_precision else 'autocast'
+            needs_model_reload = True
 
     if needs_model_reload:
-        load_model_ckpt(ckpt_file, device, req.turbo, unet_bs, precision)
+        unload_models()
+        unload_filters()
+        load_model_ckpt()
 
-    if req.use_face_correction != gfpgan_file:
-        load_model_gfpgan(req.use_face_correction)
+    if thread_data.turbo != req.turbo:
+        thread_data.turbo = req.turbo
+        thread_data.model.turbo = req.turbo
 
-    if req.use_upscale != real_esrgan_file:
-        load_model_real_esrgan(req.use_upscale)
-
-    model.cdevice = device
-    modelCS.cond_stage_model.device = device
+    # Start by cleaning memory, loading and unloading things can leave memory allocated.
+    gc()
 
     opt_prompt = req.prompt
     opt_seed = req.seed
-    opt_n_samples = req.num_outputs
     opt_n_iter = 1
-    opt_scale = req.guidance_scale
     opt_C = 4
-    opt_H = req.height
-    opt_W = req.width
     opt_f = 8
-    opt_ddim_steps = req.num_inference_steps
     opt_ddim_eta = 0.0
-    opt_strength = req.prompt_strength
-    opt_save_to_disk_path = req.save_to_disk_path
-    opt_init_img = req.init_image
-    opt_use_face_correction = req.use_face_correction
-    opt_use_upscale = req.use_upscale
-    opt_show_only_filtered = req.show_only_filtered_image
-    opt_format = req.output_format
-    opt_sampler_name = req.sampler
 
-    print(req.to_string(), '\n    device', device)
-
-    print('\n\n    Using precision:', precision)
+    print(req, '\n    device', torch.device(thread_data.device), "as", thread_data.device_name)
+    print('\n\n    Using precision:', thread_data.precision)
 
     seed_everything(opt_seed)
 
-    batch_size = opt_n_samples
+    batch_size = req.num_outputs
     prompt = opt_prompt
     assert prompt is not None
     data = [batch_size * [prompt]]
 
-    if precision == "autocast" and device != "cpu":
+    if thread_data.precision == "autocast" and thread_data.device != "cpu":
         precision_scope = autocast
     else:
         precision_scope = nullcontext
@@ -313,47 +453,47 @@ def do_mk_img(req: Request):
     else:
         handler = _img2img
 
-        init_image = load_img(req.init_image, opt_W, opt_H)
-        init_image = init_image.to(device)
+        init_image = load_img(req.init_image, req.width, req.height)
+        init_image = init_image.to(thread_data.device)
 
-        if device != "cpu" and precision == "autocast":
+        if thread_data.device != "cpu" and thread_data.precision == "autocast":
             init_image = init_image.half()
 
-        modelFS.to(device)
+        thread_data.modelFS.to(thread_data.device)
 
         init_image = repeat(init_image, '1 ... -> b ...', b=batch_size)
-        init_latent = modelFS.get_first_stage_encoding(modelFS.encode_first_stage(init_image))  # move to latent space
+        init_latent = thread_data.modelFS.get_first_stage_encoding(thread_data.modelFS.encode_first_stage(init_image))  # move to latent space
 
         if req.mask is not None:
-            mask = load_mask(req.mask, opt_W, opt_H, init_latent.shape[2], init_latent.shape[3], True).to(device)
+            mask = load_mask(req.mask, req.width, req.height, init_latent.shape[2], init_latent.shape[3], True).to(thread_data.device)
             mask = mask[0][0].unsqueeze(0).repeat(4, 1, 1).unsqueeze(0)
             mask = repeat(mask, '1 ... -> b ...', b=batch_size)
 
-            if device != "cpu" and precision == "autocast":
+            if thread_data.device != "cpu" and thread_data.precision == "autocast":
                 mask = mask.half()
 
-        move_fs_to_cpu()
+        # Send to CPU and wait until complete.
+        wait_model_move_to(thread_data.modelFS, 'cpu')
 
-        assert 0. <= opt_strength <= 1., 'can only work with strength in [0.0, 1.0]'
-        t_enc = int(opt_strength * opt_ddim_steps)
+        assert 0. <= req.prompt_strength <= 1., 'can only work with strength in [0.0, 1.0]'
+        t_enc = int(req.prompt_strength * req.num_inference_steps)
         print(f"target t_enc is {t_enc} steps")
 
-    if opt_save_to_disk_path is not None:
-        session_out_path = os.path.join(opt_save_to_disk_path, req.session_id)
-        os.makedirs(session_out_path, exist_ok=True)
+    if req.save_to_disk_path is not None:
+        session_out_path = get_session_out_path(req.save_to_disk_path, req.session_id)
     else:
         session_out_path = None
 
-    seeds = ""
     with torch.no_grad():
         for n in trange(opt_n_iter, desc="Sampling"):
             for prompts in tqdm(data, desc="data"):
 
                 with precision_scope("cuda"):
-                    modelCS.to(device)
+                    if thread_data.reduced_memory:
+                        thread_data.modelCS.to(thread_data.device)
                     uc = None
-                    if opt_scale != 1.0:
-                        uc = modelCS.get_learned_conditioning(batch_size * [req.negative_prompt])
+                    if req.guidance_scale != 1.0:
+                        uc = thread_data.modelCS.get_learned_conditioning(batch_size * [req.negative_prompt])
                     if isinstance(prompts, tuple):
                         prompts = list(prompts)
 
@@ -366,147 +506,112 @@ def do_mk_img(req: Request):
                             weight = weights[i]
                             # if not skip_normalize:
                             weight = weight / totalWeight
-                            c = torch.add(c, modelCS.get_learned_conditioning(subprompts[i]), alpha=weight)
+                            c = torch.add(c, thread_data.modelCS.get_learned_conditioning(subprompts[i]), alpha=weight)
                     else:
-                        c = modelCS.get_learned_conditioning(prompts)
+                        c = thread_data.modelCS.get_learned_conditioning(prompts)
 
-                    modelFS.to(device)
+                    if thread_data.reduced_memory:
+                        thread_data.modelFS.to(thread_data.device)
 
-                    partial_x_samples = None
-                    def img_callback(x_samples, i):
-                        nonlocal partial_x_samples
-
-                        partial_x_samples = x_samples
-
-                        if req.stream_progress_updates:
-                            n_steps = opt_ddim_steps if req.init_image is None else t_enc
-                            progress = {"step": i, "total_steps": n_steps}
-
-                            if req.stream_image_progress and i % 5 == 0:
-                                partial_images = []
-
-                                for i in range(batch_size):
-                                    x_samples_ddim = modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
-                                    x_sample = torch.clamp((x_samples_ddim + 1.0) / 2.0, min=0.0, max=1.0)
-                                    x_sample = 255.0 * rearrange(x_sample[0].cpu().numpy(), "c h w -> h w c")
-                                    x_sample = x_sample.astype(np.uint8)
-                                    img = Image.fromarray(x_sample)
-                                    buf = BytesIO()
-                                    img.save(buf, format='JPEG')
-                                    buf.seek(0)
-
-                                    del img, x_sample, x_samples_ddim
-                                    # don't delete x_samples, it is used in the code that called this callback
-
-                                    temp_images[str(req.session_id) + '/' + str(i)] = buf
-                                    partial_images.append({'path': f'/image/tmp/{req.session_id}/{i}'})
-
-                                progress['output'] = partial_images
-
-                            yield json.dumps(progress)
-
-                        if stop_processing:
-                            raise UserInitiatedStop("User requested that we stop processing")
+                    n_steps = req.num_inference_steps if req.init_image is None else t_enc
+                    img_callback = get_image_progress_generator(req, {"total_steps": n_steps})
 
                     # run the handler
                     try:
+                        print('Running handler...')
                         if handler == _txt2img:
-                            x_samples = _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, None, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed, img_callback, mask, opt_sampler_name)
+                            x_samples = _txt2img(req.width, req.height, req.num_outputs, req.num_inference_steps, req.guidance_scale, None, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed, img_callback, mask, req.sampler)
                         else:
-                            x_samples = _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed, img_callback, mask)
+                            x_samples = _img2img(init_latent, t_enc, batch_size, req.guidance_scale, c, uc, req.num_inference_steps, opt_ddim_eta, opt_seed, img_callback, mask)
 
-                        yield from x_samples
-
-                        x_samples = partial_x_samples
+                        if req.stream_progress_updates:
+                            yield from x_samples
+                        if hasattr(thread_data, 'partial_x_samples'):
+                            if thread_data.partial_x_samples is not None:
+                                x_samples = thread_data.partial_x_samples
+                            del thread_data.partial_x_samples
                     except UserInitiatedStop:
-                        if partial_x_samples is None:
+                        if not hasattr(thread_data, 'partial_x_samples'):
                             continue
+                        if thread_data.partial_x_samples is None:
+                            del thread_data.partial_x_samples
+                            continue
+                        x_samples = thread_data.partial_x_samples
+                        del thread_data.partial_x_samples
 
-                        x_samples = partial_x_samples
-
-                    print("saving images")
+                    print("decoding images")
+                    img_data = [None] * batch_size
                     for i in range(batch_size):
-
-                        x_samples_ddim = modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
+                        x_samples_ddim = thread_data.modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
                         x_sample = torch.clamp((x_samples_ddim + 1.0) / 2.0, min=0.0, max=1.0)
                         x_sample = 255.0 * rearrange(x_sample[0].cpu().numpy(), "c h w -> h w c")
                         x_sample = x_sample.astype(np.uint8)
-                        img = Image.fromarray(x_sample)
+                        img_data[i] = x_sample
+                    del x_samples, x_samples_ddim, x_sample
 
-                        has_filters =   (opt_use_face_correction is not None and opt_use_face_correction.startswith('GFPGAN')) or \
-                                        (opt_use_upscale is not None and opt_use_upscale.startswith('RealESRGAN'))
+                    if thread_data.reduced_memory:
+                        # Send to CPU and wait until complete.
+                        wait_model_move_to(thread_data.modelFS, 'cpu')
 
-                        return_orig_img = not has_filters or not opt_show_only_filtered
+                    print("saving images")
+                    for i in range(batch_size):
+                        img = Image.fromarray(img_data[i])
+                        img_id = base64.b64encode(int(time.time()+i).to_bytes(8, 'big')).decode() # Generate unique ID based on time.
+                        img_id = img_id.translate({43:None, 47:None, 61:None})[-8:] # Remove + / = and keep last 8 chars.
 
-                        if stop_processing:
+                        has_filters =   (req.use_face_correction is not None and req.use_face_correction.startswith('GFPGAN')) or \
+                                        (req.use_upscale is not None and req.use_upscale.startswith('RealESRGAN'))
+
+                        return_orig_img = not has_filters or not req.show_only_filtered_image
+
+                        if thread_data.stop_processing:
                             return_orig_img = True
 
-                        if opt_save_to_disk_path is not None:
-                            prompt_flattened = filename_regex.sub('_', prompts[0])
-                            prompt_flattened = prompt_flattened[:50]
-
-                            img_id = str(uuid.uuid4())[-8:]
-
-                            file_path = f"{prompt_flattened}_{img_id}"
-                            img_out_path = os.path.join(session_out_path, f"{file_path}.{opt_format}")
-                            meta_out_path = os.path.join(session_out_path, f"{file_path}.txt")
-
+                        if req.save_to_disk_path is not None:
                             if return_orig_img:
+                                img_out_path = get_base_path(req.save_to_disk_path, req.session_id, prompts[0], img_id, req.output_format)
                                 save_image(img, img_out_path)
-
-                            save_metadata(meta_out_path, prompts, opt_seed, opt_W, opt_H, opt_ddim_steps, opt_scale, opt_strength, opt_use_face_correction, opt_use_upscale, opt_sampler_name, req.negative_prompt, ckpt_file)
+                            meta_out_path = get_base_path(req.save_to_disk_path, req.session_id, prompts[0], img_id, 'txt')
+                            save_metadata(meta_out_path, req, prompts[0], opt_seed)
 
                         if return_orig_img:
-                            img_data = img_to_base64_str(img, opt_format)
-                            res_image_orig = ResponseImage(data=img_data, seed=opt_seed)
+                            img_str = img_to_base64_str(img, req.output_format)
+                            res_image_orig = ResponseImage(data=img_str, seed=opt_seed)
                             res.images.append(res_image_orig)
 
-                            if opt_save_to_disk_path is not None:
+                            if req.save_to_disk_path is not None:
                                 res_image_orig.path_abs = img_out_path
-
                         del img
 
-                        if has_filters and not stop_processing:
-                            print('Applying filters..')
-
-                            gc()
+                        if has_filters and not thread_data.stop_processing:
                             filters_applied = []
-
-                            if opt_use_face_correction:
-                                _, _, output = model_gfpgan.enhance(x_sample[:,:,::-1], has_aligned=False, only_center_face=False, paste_back=True)
-                                x_sample = output[:,:,::-1]
-                                filters_applied.append(opt_use_face_correction)
-
-                            if opt_use_upscale:
-                                output, _ = model_real_esrgan.enhance(x_sample[:,:,::-1])
-                                x_sample = output[:,:,::-1]
-                                filters_applied.append(opt_use_upscale)
-
-                            filtered_image = Image.fromarray(x_sample)
-
-                            filtered_img_data = img_to_base64_str(filtered_image, opt_format)
-                            res_image_filtered = ResponseImage(data=filtered_img_data, seed=opt_seed)
-                            res.images.append(res_image_filtered)
-
-                            filters_applied = "_".join(filters_applied)
-
-                            if opt_save_to_disk_path is not None:
-                                filtered_img_out_path = os.path.join(session_out_path, f"{file_path}_{filters_applied}.{opt_format}")
-                                save_image(filtered_image, filtered_img_out_path)
-                                res_image_filtered.path_abs = filtered_img_out_path
-
-                            del filtered_image
-
-                        seeds += str(opt_seed) + ","
+                            if req.use_face_correction:
+                                img_data[i] = apply_filters('gfpgan', img_data[i], req.use_face_correction)
+                                filters_applied.append(req.use_face_correction)
+                            if req.use_upscale:
+                                img_data[i] = apply_filters('real_esrgan', img_data[i], req.use_upscale)
+                                filters_applied.append(req.use_upscale)
+                            if (len(filters_applied) > 0):
+                                filtered_image = Image.fromarray(img_data[i])
+                                filtered_img_data = img_to_base64_str(filtered_image, req.output_format)
+                                response_image = ResponseImage(data=filtered_img_data, seed=opt_seed)
+                                res.images.append(response_image)
+                                if req.save_to_disk_path is not None:
+                                    filtered_img_out_path = get_base_path(req.save_to_disk_path, req.session_id, prompts[0], img_id, req.output_format, "_".join(filters_applied))
+                                    save_image(filtered_image, filtered_img_out_path)
+                                    response_image.path_abs = filtered_img_out_path
+                                del filtered_image
+                        # Filter Applied, move to next seed
                         opt_seed += 1
 
-                    move_fs_to_cpu()
+                    # if thread_data.reduced_memory:
+                    #     unload_filters()
+                    del img_data
                     gc()
-                    del x_samples, x_samples_ddim, x_sample
-                    print("memory_final = ", torch.cuda.memory_allocated() / 1e6)
+                    if thread_data.device != 'cpu':
+                        print(f'memory_final = {round(torch.cuda.memory_allocated(thread_data.device) / 1e6, 2)}Mb')
 
     print('Task completed')
-
     yield json.dumps(res.json())
 
 def save_image(img, img_out_path):
@@ -515,11 +620,23 @@ def save_image(img, img_out_path):
     except:
         print('could not save the file', traceback.format_exc())
 
-def save_metadata(meta_out_path, prompts, opt_seed, opt_W, opt_H, opt_ddim_steps, opt_scale, opt_prompt_strength, opt_correct_face, opt_upscale, sampler_name, negative_prompt, ckpt_file):
-    metadata = f"{prompts[0]}\nWidth: {opt_W}\nHeight: {opt_H}\nSeed: {opt_seed}\nSteps: {opt_ddim_steps}\nGuidance Scale: {opt_scale}\nPrompt Strength: {opt_prompt_strength}\nUse Face Correction: {opt_correct_face}\nUse Upscaling: {opt_upscale}\nSampler: {sampler_name}\nNegative Prompt: {negative_prompt}\nStable Diffusion Model: {ckpt_file + '.ckpt'}"
-
+def save_metadata(meta_out_path, req, prompt, opt_seed):
+    metadata = f'''{prompt}
+Width: {req.width}
+Height: {req.height}
+Seed: {opt_seed}
+Steps: {req.num_inference_steps}
+Guidance Scale: {req.guidance_scale}
+Prompt Strength: {req.prompt_strength}
+Use Face Correction: {req.use_face_correction}
+Use Upscaling: {req.use_upscale}
+Sampler: {req.sampler}
+Negative Prompt: {req.negative_prompt}
+Stable Diffusion model: {req.use_stable_diffusion_model + '.ckpt'}
+VAE model: {req.use_vae_model}
+'''
     try:
-        with open(meta_out_path, 'w') as f:
+        with open(meta_out_path, 'w', encoding='utf-8') as f:
             f.write(metadata)
     except:
         print('could not save the file', traceback.format_exc())
@@ -527,16 +644,13 @@ def save_metadata(meta_out_path, prompts, opt_seed, opt_W, opt_H, opt_ddim_steps
 def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed, img_callback, mask, sampler_name):
     shape = [opt_n_samples, opt_C, opt_H // opt_f, opt_W // opt_f]
 
-    if device != "cpu":
-        mem = torch.cuda.memory_allocated() / 1e6
-        modelCS.to("cpu")
-        while torch.cuda.memory_allocated() / 1e6 >= mem:
-            time.sleep(1)
+    # Send to CPU and wait until complete.
+    wait_model_move_to(thread_data.modelCS, 'cpu')
 
     if sampler_name == 'ddim':
-        model.make_schedule(ddim_num_steps=opt_ddim_steps, ddim_eta=opt_ddim_eta, verbose=False)
+        thread_data.model.make_schedule(ddim_num_steps=opt_ddim_steps, ddim_eta=opt_ddim_eta, verbose=False)
 
-    samples_ddim = model.sample(
+    samples_ddim = thread_data.model.sample(
         S=opt_ddim_steps,
         conditioning=c,
         seed=opt_seed,
@@ -550,14 +664,13 @@ def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code,
         mask=mask,
         sampler = sampler_name,
     )
-
     yield from samples_ddim
 
 def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed, img_callback, mask):
     # encode (scaled latent)
-    z_enc = model.stochastic_encode(
+    z_enc = thread_data.model.stochastic_encode(
         init_latent,
-        torch.tensor([t_enc] * batch_size).to(device),
+        torch.tensor([t_enc] * batch_size).to(thread_data.device),
         opt_seed,
         opt_ddim_eta,
         opt_ddim_steps,
@@ -565,7 +678,7 @@ def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, o
     x_T = None if mask is None else init_latent
 
     # decode it
-    samples_ddim = model.sample(
+    samples_ddim = thread_data.model.sample(
         t_enc,
         c,
         z_enc,
@@ -576,20 +689,12 @@ def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, o
         x_T=x_T,
         sampler = 'ddim'
     )
-
     yield from samples_ddim
 
-def move_fs_to_cpu():
-    if device != "cpu":
-        mem = torch.cuda.memory_allocated() / 1e6
-        modelFS.to("cpu")
-        while torch.cuda.memory_allocated() / 1e6 >= mem:
-            time.sleep(1)
-
 def gc():
-    if device == 'cpu':
+    gc_collect()
+    if thread_data.device == 'cpu':
         return
-
     torch.cuda.empty_cache()
     torch.cuda.ipc_collect()
 
@@ -599,7 +704,6 @@ def chunk(it, size):
     it = iter(it)
     return iter(lambda: tuple(islice(it, size)), ())
 
-
 def load_model_from_config(ckpt, verbose=False):
     print(f"Loading model from {ckpt}")
     pl_sd = torch.load(ckpt, map_location="cpu")
@@ -657,12 +761,18 @@ def img_to_base64_str(img, output_format="PNG"):
     img.save(buffered, format=output_format)
     buffered.seek(0)
     img_byte = buffered.getvalue()
-    img_str = "data:image/png;base64," + base64.b64encode(img_byte).decode()
+    mime_type = "image/png" if output_format.lower() == "png" else "image/jpeg"
+    img_str = f"data:{mime_type};base64," + base64.b64encode(img_byte).decode()
     return img_str
 
-def base64_str_to_img(img_str):
-    img_str = img_str[len("data:image/png;base64,"):]
+def base64_str_to_buffer(img_str):
+    mime_type = "image/png" if img_str.startswith("data:image/png;") else "image/jpeg"
+    img_str = img_str[len(f"data:{mime_type};base64,"):]
     data = base64.b64decode(img_str)
     buffered = BytesIO(data)
+    return buffered
+
+def base64_str_to_img(img_str):
+    buffered = base64_str_to_buffer(img_str)
     img = Image.open(buffered)
     return img

ui/sd_internal/task_manager.py

@@ -0,0 +1,548 @@
+"""task_manager.py: manage tasks dispatching and render threads.
+Notes:
+    render_threads should be the only hard reference held by the manager to the threads.
+    Use weak_thread_data to store all other data using weak keys.
+    This will allow for garbage collection after the thread dies.
+"""
+import json
+import traceback
+
+TASK_TTL = 15 * 60 # seconds, Discard last session's task timeout
+
+import torch
+import queue, threading, time, weakref
+from typing import Any, Generator, Hashable, Optional, Union
+
+from pydantic import BaseModel
+from sd_internal import Request, Response, runtime, device_manager
+
+THREAD_NAME_PREFIX = 'Runtime-Render/'
+ERR_LOCK_FAILED = ' failed to acquire lock within timeout.'
+LOCK_TIMEOUT = 15 # Maximum locking time in seconds before failing a task.
+# It's better to get an exception than a deadlock... ALWAYS use timeout in critical paths.
+
+DEVICE_START_TIMEOUT = 60 # seconds - Maximum time to wait for a render device to init.
+
+class SymbolClass(type): # Print nicely formatted Symbol names.
+    def __repr__(self): return self.__qualname__
+    def __str__(self): return self.__name__
+class Symbol(metaclass=SymbolClass): pass
+
+class ServerStates:
+    class Init(Symbol): pass
+    class LoadingModel(Symbol): pass
+    class Online(Symbol): pass
+    class Rendering(Symbol): pass
+    class Unavailable(Symbol): pass
+
+class RenderTask(): # Task with output queue and completion lock.
+    def __init__(self, req: Request):
+        self.request: Request = req # Initial Request
+        self.response: Any = None # Copy of the last reponse
+        self.render_device = None # Select the task affinity. (Not used to change active devices).
+        self.temp_images:list = [None] * req.num_outputs * (1 if req.show_only_filtered_image else 2)
+        self.error: Exception = None
+        self.lock: threading.Lock = threading.Lock() # Locks at task start and unlocks when task is completed
+        self.buffer_queue: queue.Queue = queue.Queue() # Queue of JSON string segments
+    async def read_buffer_generator(self):
+        try:
+            while not self.buffer_queue.empty():
+                res = self.buffer_queue.get(block=False)
+                self.buffer_queue.task_done()
+                yield res
+        except queue.Empty as e: yield
+
+# defaults from https://huggingface.co/blog/stable_diffusion
+class ImageRequest(BaseModel):
+    session_id: str = "session"
+    prompt: str = ""
+    negative_prompt: str = ""
+    init_image: str = None # base64
+    mask: str = None # base64
+    num_outputs: int = 1
+    num_inference_steps: int = 50
+    guidance_scale: float = 7.5
+    width: int = 512
+    height: int = 512
+    seed: int = 42
+    prompt_strength: float = 0.8
+    sampler: str = None # "ddim", "plms", "heun", "euler", "euler_a", "dpm2", "dpm2_a", "lms"
+    # allow_nsfw: bool = False
+    save_to_disk_path: str = None
+    turbo: bool = True
+    use_cpu: bool = False ##TODO Remove after UI and plugins transition.
+    render_device: str = None # Select the task affinity. (Not used to change active devices).
+    use_full_precision: bool = False
+    use_face_correction: str = None # or "GFPGANv1.3"
+    use_upscale: str = None # or "RealESRGAN_x4plus" or "RealESRGAN_x4plus_anime_6B"
+    use_stable_diffusion_model: str = "sd-v1-4"
+    use_vae_model: str = None
+    show_only_filtered_image: bool = False
+    output_format: str = "jpeg" # or "png"
+
+    stream_progress_updates: bool = False
+    stream_image_progress: bool = False
+
+class FilterRequest(BaseModel):
+    session_id: str = "session"
+    model: str = None
+    name: str = ""
+    init_image: str = None # base64
+    width: int = 512
+    height: int = 512
+    save_to_disk_path: str = None
+    turbo: bool = True
+    render_device: str = None
+    use_full_precision: bool = False
+    output_format: str = "jpeg" # or "png"
+
+# Temporary cache to allow to query tasks results for a short time after they are completed.
+class TaskCache():
+    def __init__(self):
+        self._base = dict()
+        self._lock: threading.Lock = threading.Lock()
+    def _get_ttl_time(self, ttl: int) -> int:
+        return int(time.time()) + ttl
+    def _is_expired(self, timestamp: int) -> bool:
+        return int(time.time()) >= timestamp
+    def clean(self) -> None:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.clean' + ERR_LOCK_FAILED)
+        try:
+            # Create a list of expired keys to delete
+            to_delete = []
+            for key in self._base:
+                ttl, _ = self._base[key]
+                if self._is_expired(ttl):
+                    to_delete.append(key)
+            # Remove Items
+            for key in to_delete:
+                del self._base[key]
+                print(f'Session {key} expired. Data removed.')
+        finally:
+            self._lock.release()
+    def clear(self) -> None:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.clear' + ERR_LOCK_FAILED)
+        try: self._base.clear()
+        finally: self._lock.release()
+    def delete(self, key: Hashable) -> bool:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.delete' + ERR_LOCK_FAILED)
+        try:
+            if key not in self._base:
+                return False
+            del self._base[key]
+            return True
+        finally:
+            self._lock.release()
+    def keep(self, key: Hashable, ttl: int) -> bool:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.keep' + ERR_LOCK_FAILED)
+        try:
+            if key in self._base:
+                _, value = self._base.get(key)
+                self._base[key] = (self._get_ttl_time(ttl), value)
+                return True
+            return False
+        finally:
+            self._lock.release()
+    def put(self, key: Hashable, value: Any, ttl: int) -> bool:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.put' + ERR_LOCK_FAILED)
+        try:
+            self._base[key] = (
+                self._get_ttl_time(ttl), value
+            )
+        except Exception as e:
+            print(str(e))
+            print(traceback.format_exc())
+            return False
+        else:
+            return True
+        finally:
+            self._lock.release()
+    def tryGet(self, key: Hashable) -> Any:
+        if not self._lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('TaskCache.tryGet' + ERR_LOCK_FAILED)
+        try:
+            ttl, value = self._base.get(key, (None, None))
+            if ttl is not None and self._is_expired(ttl):
+                print(f'Session {key} expired. Discarding data.')
+                del self._base[key]
+                return None
+            return value
+        finally:
+            self._lock.release()
+
+manager_lock = threading.RLock()
+render_threads = []
+current_state = ServerStates.Init
+current_state_error:Exception = None
+current_model_path = None
+current_vae_path = None
+tasks_queue = []
+task_cache = TaskCache()
+default_model_to_load = None
+default_vae_to_load = None
+weak_thread_data = weakref.WeakKeyDictionary()
+
+def preload_model(ckpt_file_path=None, vae_file_path=None):
+    global current_state, current_state_error, current_model_path, current_vae_path
+    if ckpt_file_path == None:
+        ckpt_file_path = default_model_to_load
+    if vae_file_path == None:
+        vae_file_path = default_vae_to_load
+    if ckpt_file_path == current_model_path and vae_file_path == current_vae_path:
+        return
+    current_state = ServerStates.LoadingModel
+    try:
+        from . import runtime
+        runtime.thread_data.ckpt_file = ckpt_file_path
+        runtime.thread_data.vae_file = vae_file_path
+        runtime.load_model_ckpt()
+        current_model_path = ckpt_file_path
+        current_vae_path = vae_file_path
+        current_state_error = None
+        current_state = ServerStates.Online
+    except Exception as e:
+        current_model_path = None
+        current_vae_path = None
+        current_state_error = e
+        current_state = ServerStates.Unavailable
+        print(traceback.format_exc())
+
+def thread_get_next_task():
+    from . import runtime
+    if not manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT):
+        print('Render thread on device', runtime.thread_data.device, 'failed to acquire manager lock.')
+        return None
+    if len(tasks_queue) <= 0:
+        manager_lock.release()
+        return None
+    task = None
+    try:  # Select a render task.
+        for queued_task in tasks_queue:
+            if queued_task.request.use_face_correction and runtime.thread_data.device == 'cpu' and is_alive() == 1:
+                queued_task.error = Exception('The CPU cannot be used to run this task currently. Please remove "Fix incorrect faces" from Image Settings and try again.')
+                task = queued_task
+                break
+            if queued_task.render_device and runtime.thread_data.device != queued_task.render_device:
+                # Is asking for a specific render device.
+                if is_alive(queued_task.render_device) > 0:
+                    continue  # requested device alive, skip current one.
+                else:
+                    # Requested device is not active, return error to UI.
+                    queued_task.error = Exception(queued_task.render_device + ' is not currently active.')
+                    task = queued_task
+                    break
+            if not queued_task.render_device and runtime.thread_data.device == 'cpu' and is_alive() > 1:
+                # not asking for any specific devices, cpu want to grab task but other render devices are alive.
+                continue  # Skip Tasks, don't run on CPU unless there is nothing else or user asked for it.
+            task = queued_task
+            break
+        if task is not None:
+            del tasks_queue[tasks_queue.index(task)]
+        return task
+    finally:
+        manager_lock.release()
+
+def thread_render(device):
+    global current_state, current_state_error, current_model_path, current_vae_path
+    from . import runtime
+    try:
+        runtime.thread_init(device)
+    except Exception as e:
+        print(traceback.format_exc())
+        weak_thread_data[threading.current_thread()] = {
+            'error': e
+        }
+        return
+    weak_thread_data[threading.current_thread()] = {
+        'device': runtime.thread_data.device,
+        'device_name': runtime.thread_data.device_name,
+        'alive': True
+    }
+    if runtime.thread_data.device != 'cpu' or is_alive() == 1:
+        preload_model()
+        current_state = ServerStates.Online
+    while True:
+        task_cache.clean()
+        if not weak_thread_data[threading.current_thread()]['alive']:
+            print(f'Shutting down thread for device {runtime.thread_data.device}')
+            runtime.unload_models()
+            runtime.unload_filters()
+            return
+        if isinstance(current_state_error, SystemExit):
+            current_state = ServerStates.Unavailable
+            return
+        task = thread_get_next_task()
+        if task is None:
+            time.sleep(1)
+            continue
+        if task.error is not None:
+            print(task.error)
+            task.response = {"status": 'failed', "detail": str(task.error)}
+            task.buffer_queue.put(json.dumps(task.response))
+            continue
+        if current_state_error:
+            task.error = current_state_error
+            task.response = {"status": 'failed', "detail": str(task.error)}
+            task.buffer_queue.put(json.dumps(task.response))
+            continue
+        print(f'Session {task.request.session_id} starting task {id(task)} on {runtime.thread_data.device_name}')
+        if not task.lock.acquire(blocking=False): raise Exception('Got locked task from queue.')
+        try:
+            if runtime.thread_data.device == 'cpu' and is_alive() > 1:
+                # CPU is not the only device. Keep track of active time to unload resources later.
+                runtime.thread_data.lastActive = time.time()
+            # Open data generator.
+            res = runtime.mk_img(task.request)
+            if current_model_path == task.request.use_stable_diffusion_model:
+                current_state = ServerStates.Rendering
+            else:
+                current_state = ServerStates.LoadingModel
+            # Start reading from generator.
+            dataQueue = None
+            if task.request.stream_progress_updates:
+                dataQueue = task.buffer_queue
+            for result in res:
+                if current_state == ServerStates.LoadingModel:
+                    current_state = ServerStates.Rendering
+                    current_model_path = task.request.use_stable_diffusion_model
+                    current_vae_path = task.request.use_vae_model
+                if isinstance(current_state_error, SystemExit) or isinstance(current_state_error, StopAsyncIteration) or isinstance(task.error, StopAsyncIteration):
+                    runtime.thread_data.stop_processing = True
+                    if isinstance(current_state_error, StopAsyncIteration):
+                        task.error = current_state_error
+                        current_state_error = None
+                        print(f'Session {task.request.session_id} sent cancel signal for task {id(task)}')
+                if dataQueue:
+                    dataQueue.put(result)
+                if isinstance(result, str):
+                    result = json.loads(result)
+                task.response = result
+                if 'output' in result:
+                    for out_obj in result['output']:
+                        if 'path' in out_obj:
+                            img_id = out_obj['path'][out_obj['path'].rindex('/') + 1:]
+                            task.temp_images[int(img_id)] = runtime.thread_data.temp_images[out_obj['path'][11:]]
+                        elif 'data' in out_obj:
+                            buf = runtime.base64_str_to_buffer(out_obj['data'])
+                            task.temp_images[result['output'].index(out_obj)] = buf
+                # Before looping back to the generator, mark cache as still alive.
+                task_cache.keep(task.request.session_id, TASK_TTL)
+        except Exception as e:
+            task.error = e
+            print(traceback.format_exc())
+            continue
+        finally:
+            # Task completed
+            task.lock.release()
+        task_cache.keep(task.request.session_id, TASK_TTL)
+        if isinstance(task.error, StopAsyncIteration):
+            print(f'Session {task.request.session_id} task {id(task)} cancelled!')
+        elif task.error is not None:
+            print(f'Session {task.request.session_id} task {id(task)} failed!')
+        else:
+            print(f'Session {task.request.session_id} task {id(task)} completed by {runtime.thread_data.device_name}.')
+        current_state = ServerStates.Online
+
+def get_cached_task(session_id:str, update_ttl:bool=False):
+    # By calling keep before tryGet, wont discard if was expired.
+    if update_ttl and not task_cache.keep(session_id, TASK_TTL):
+        # Failed to keep task, already gone.
+        return None
+    return task_cache.tryGet(session_id)
+
+def get_devices():
+    devices = {
+        'all': {},
+        'active': {},
+    }
+
+    def get_device_info(device):
+        if device == 'cpu':
+            return {'name': device_manager.get_processor_name()}
+            
+        mem_free, mem_total = torch.cuda.mem_get_info(device)
+        mem_free /= float(10**9)
+        mem_total /= float(10**9)
+
+        return {
+            'name': torch.cuda.get_device_name(device),
+            'mem_free': mem_free,
+            'mem_total': mem_total,
+        }
+
+    # list the compatible devices
+    gpu_count = torch.cuda.device_count()
+    for device in range(gpu_count):
+        device = f'cuda:{device}'
+        if not device_manager.is_device_compatible(device):
+            continue
+
+        devices['all'].update({device: get_device_info(device)})
+
+    devices['all'].update({'cpu': get_device_info('cpu')})
+
+    # list the activated devices
+    if not manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('get_devices' + ERR_LOCK_FAILED)
+    try:
+        for rthread in render_threads:
+            if not rthread.is_alive():
+                continue
+            weak_data = weak_thread_data.get(rthread)
+            if not weak_data or not 'device' in weak_data or not 'device_name' in weak_data:
+                continue
+            device = weak_data['device']
+            devices['active'].update({device: get_device_info(device)})
+    finally:
+        manager_lock.release()
+
+    return devices
+
+def is_alive(device=None):
+    if not manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('is_alive' + ERR_LOCK_FAILED)
+    nbr_alive = 0
+    try:
+        for rthread in render_threads:
+            if device is not None:
+                weak_data = weak_thread_data.get(rthread)
+                if weak_data is None or not 'device' in weak_data or weak_data['device'] is None:
+                    continue
+                thread_device = weak_data['device']
+                if thread_device != device:
+                    continue
+            if rthread.is_alive():
+                nbr_alive += 1
+        return nbr_alive
+    finally:
+        manager_lock.release()
+
+def start_render_thread(device):
+    if not manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('start_render_thread' + ERR_LOCK_FAILED)
+    print('Start new Rendering Thread on device', device)
+    try:
+        rthread = threading.Thread(target=thread_render, kwargs={'device': device})
+        rthread.daemon = True
+        rthread.name = THREAD_NAME_PREFIX + device
+        rthread.start()
+        render_threads.append(rthread)
+    finally:
+        manager_lock.release()
+    timeout = DEVICE_START_TIMEOUT
+    while not rthread.is_alive() or not rthread in weak_thread_data or not 'device' in weak_thread_data[rthread]:
+        if rthread in weak_thread_data and 'error' in weak_thread_data[rthread]:
+            print(rthread, device, 'error:', weak_thread_data[rthread]['error'])
+            return False
+        if timeout <= 0:
+            return False
+        timeout -= 1
+        time.sleep(1)
+    return True
+
+def stop_render_thread(device):
+    try:
+        device_manager.validate_device_id(device, log_prefix='stop_render_thread')
+    except:
+        print(traceback.format_exec())
+        return False
+
+    if not manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT): raise Exception('stop_render_thread' + ERR_LOCK_FAILED)
+    print('Stopping Rendering Thread on device', device)
+
+    try:
+        thread_to_remove = None
+        for rthread in render_threads:
+            weak_data = weak_thread_data.get(rthread)
+            if weak_data is None or not 'device' in weak_data or weak_data['device'] is None:
+                continue
+            thread_device = weak_data['device']
+            if thread_device == device:
+                weak_data['alive'] = False
+                thread_to_remove = rthread
+                break
+        if thread_to_remove is not None:
+            render_threads.remove(rthread)
+            return True
+    finally:
+        manager_lock.release()
+
+    return False
+
+def update_render_threads(render_devices, active_devices):
+    devices_to_start, devices_to_stop = device_manager.get_device_delta(render_devices, active_devices)
+    print('devices_to_start', devices_to_start)
+    print('devices_to_stop', devices_to_stop)
+
+    for device in devices_to_stop:
+        if is_alive(device) <= 0:
+            print(device, 'is not alive')
+            continue
+        if not stop_render_thread(device):
+            print(device, 'could not stop render thread')
+
+    for device in devices_to_start:
+        if is_alive(device) >= 1:
+            print(device, 'already registered.')
+            continue
+        if not start_render_thread(device):
+            print(device, 'failed to start.')
+
+    if is_alive() <= 0: # No running devices, probably invalid user config.
+        raise EnvironmentError('ERROR: No active render devices! Please verify the "render_devices" value in config.json')
+
+    print('active devices', get_devices()['active'])
+
+def shutdown_event(): # Signal render thread to close on shutdown
+    global current_state_error
+    current_state_error = SystemExit('Application shutting down.')
+
+def render(req : ImageRequest):
+    if is_alive() <= 0: # Render thread is dead
+        raise ChildProcessError('Rendering thread has died.')
+    # Alive, check if task in cache
+    task = task_cache.tryGet(req.session_id)
+    if task and not task.response and not task.error and not task.lock.locked():
+        # Unstarted task pending, deny queueing more than one.
+        raise ConnectionRefusedError(f'Session {req.session_id} has an already pending task.')
+    #
+    from . import runtime
+    r = Request()
+    r.session_id = req.session_id
+    r.prompt = req.prompt
+    r.negative_prompt = req.negative_prompt
+    r.init_image = req.init_image
+    r.mask = req.mask
+    r.num_outputs = req.num_outputs
+    r.num_inference_steps = req.num_inference_steps
+    r.guidance_scale = req.guidance_scale
+    r.width = req.width
+    r.height = req.height
+    r.seed = req.seed
+    r.prompt_strength = req.prompt_strength
+    r.sampler = req.sampler
+    # r.allow_nsfw = req.allow_nsfw
+    r.turbo = req.turbo
+    r.use_full_precision = req.use_full_precision
+    r.save_to_disk_path = req.save_to_disk_path
+    r.use_upscale: str = req.use_upscale
+    r.use_face_correction = req.use_face_correction
+    r.use_stable_diffusion_model = req.use_stable_diffusion_model
+    r.use_vae_model = req.use_vae_model
+    r.show_only_filtered_image = req.show_only_filtered_image
+    r.output_format = req.output_format
+
+    r.stream_progress_updates = True # the underlying implementation only supports streaming
+    r.stream_image_progress = req.stream_image_progress
+
+    if not req.stream_progress_updates:
+        r.stream_image_progress = False
+
+    new_task = RenderTask(r)
+
+    if task_cache.put(r.session_id, new_task, TASK_TTL):
+        # Use twice the normal timeout for adding user requests.
+        # Tries to force task_cache.put to fail before tasks_queue.put would. 
+        if manager_lock.acquire(blocking=True, timeout=LOCK_TIMEOUT * 2):
+            try:
+                tasks_queue.append(new_task)
+                return new_task
+            finally:
+                manager_lock.release()
+    raise RuntimeError('Failed to add task to cache.')

ui/server.py

@@ -1,3 +1,7 @@
+"""server.py: FastAPI SD-UI Web Host.
+Notes:
+    async endpoints always run on the main thread. Without they run on the thread pool.
+"""
 import json
 import traceback
 
@@ -12,309 +16,357 @@ sys.path.append(os.path.dirname(SD_UI_DIR))
 
 CONFIG_DIR = os.path.abspath(os.path.join(SD_UI_DIR, '..', 'scripts'))
 MODELS_DIR = os.path.abspath(os.path.join(SD_DIR, '..', 'models'))
+UI_PLUGINS_DIR = os.path.abspath(os.path.join(SD_DIR, '..', 'plugins', 'ui'))
 
 OUTPUT_DIRNAME = "Stable Diffusion UI" # in the user's home folder
+TASK_TTL = 15 * 60 # Discard last session's task timeout
+APP_CONFIG_DEFAULTS = {
+    # auto: selects the cuda device with the most free memory, cuda: use the currently active cuda device.
+    'render_devices': 'auto', # valid entries: 'auto', 'cpu' or 'cuda:N' (where N is a GPU index)
+    'update_branch': 'main',
+    'ui': {
+        'open_browser_on_start': True,
+    },
+}
+APP_CONFIG_DEFAULT_MODELS = [
+    # needed to support the legacy installations
+    'custom-model', # Check if user has a custom model, use it first.
+    'sd-v1-4', # Default fallback.
+]
 
 from fastapi import FastAPI, HTTPException
 from fastapi.staticfiles import StaticFiles
-from starlette.responses import FileResponse, StreamingResponse
+from starlette.responses import FileResponse, JSONResponse, StreamingResponse
 from pydantic import BaseModel
 import logging
+#import queue, threading, time
+from typing import Any, Generator, Hashable, List, Optional, Union
 
-from sd_internal import Request, Response
+from sd_internal import Request, Response, task_manager
 
 app = FastAPI()
 
-model_loaded = False
-model_is_loading = False
-
 modifiers_cache = None
 outpath = os.path.join(os.path.expanduser("~"), OUTPUT_DIRNAME)
 
+os.makedirs(UI_PLUGINS_DIR, exist_ok=True)
+
 # don't show access log entries for URLs that start with the given prefix
-ACCESS_LOG_SUPPRESS_PATH_PREFIXES = ['/ping', '/modifier-thumbnails']
+ACCESS_LOG_SUPPRESS_PATH_PREFIXES = ['/ping', '/image', '/modifier-thumbnails']
 
-app.mount('/media', StaticFiles(directory=os.path.join(SD_UI_DIR, 'media/')), name="media")
+NOCACHE_HEADERS={"Cache-Control": "no-cache, no-store, must-revalidate", "Pragma": "no-cache", "Expires": "0"}
 
-# defaults from https://huggingface.co/blog/stable_diffusion
-class ImageRequest(BaseModel):
-    session_id: str = "session"
-    prompt: str = ""
-    negative_prompt: str = ""
-    init_image: str = None # base64
-    mask: str = None # base64
-    num_outputs: int = 1
-    num_inference_steps: int = 50
-    guidance_scale: float = 7.5
-    width: int = 512
-    height: int = 512
-    seed: int = 42
-    prompt_strength: float = 0.8
-    sampler: str = None # "ddim", "plms", "heun", "euler", "euler_a", "dpm2", "dpm2_a", "lms"
-    # allow_nsfw: bool = False
-    save_to_disk_path: str = None
-    turbo: bool = True
-    use_cpu: bool = False
-    use_full_precision: bool = False
-    use_face_correction: str = None # or "GFPGANv1.3"
-    use_upscale: str = None # or "RealESRGAN_x4plus" or "RealESRGAN_x4plus_anime_6B"
-    use_stable_diffusion_model: str = "sd-v1-4"
-    show_only_filtered_image: bool = False
-    output_format: str = "jpeg" # or "png"
+app.mount('/media', StaticFiles(directory=os.path.join(SD_UI_DIR, 'media')), name="media")
+app.mount('/plugins', StaticFiles(directory=UI_PLUGINS_DIR), name="plugins")
 
-    stream_progress_updates: bool = False
-    stream_image_progress: bool = False
-
-class SetAppConfigRequest(BaseModel):
-    update_branch: str = "main"
-
-@app.get('/')
-def read_root():
-    headers = {"Cache-Control": "no-cache, no-store, must-revalidate", "Pragma": "no-cache", "Expires": "0"}
-    return FileResponse(os.path.join(SD_UI_DIR, 'index.html'), headers=headers)
-
-@app.get('/ping')
-async def ping():
-    global model_loaded, model_is_loading
-
-    try:
-        if model_loaded:
-            return {'OK'}
-
-        if model_is_loading:
-            return {'ERROR'}
-
-        model_is_loading = True
-
-        from sd_internal import runtime
-
-        runtime.load_model_ckpt(ckpt_to_use=get_initial_model_to_load())
-
-        model_loaded = True
-        model_is_loading = False
-
-        return {'OK'}
-    except Exception as e:
-        print(traceback.format_exc())
-        return HTTPException(status_code=500, detail=str(e))
-
-# needs to support the legacy installations
-def get_initial_model_to_load():
-    custom_weight_path = os.path.join(SD_DIR, 'custom-model.ckpt')
-    ckpt_to_use = "sd-v1-4" if not os.path.exists(custom_weight_path) else "custom-model"
-
-    ckpt_to_use = os.path.join(SD_DIR, ckpt_to_use)
-
-    config = getConfig()
-    if 'model' in config and 'stable-diffusion' in config['model']:
-        model_name = config['model']['stable-diffusion']
-        model_path = resolve_model_to_use(model_name)
-
-        if os.path.exists(model_path + '.ckpt'):
-            ckpt_to_use = model_path
-        else:
-            print('Could not find the configured custom model at:', model_path + '.ckpt', '. Using the default one:', ckpt_to_use + '.ckpt')
-
-    return ckpt_to_use
-
-def resolve_model_to_use(model_name):
-    if model_name in ('sd-v1-4', 'custom-model'):
-        model_path = os.path.join(MODELS_DIR, 'stable-diffusion', model_name)
-
-        legacy_model_path = os.path.join(SD_DIR, model_name)
-        if not os.path.exists(model_path + '.ckpt') and os.path.exists(legacy_model_path + '.ckpt'):
-            model_path = legacy_model_path
-    else:
-        model_path = os.path.join(MODELS_DIR, 'stable-diffusion', model_name)
-
-    return model_path
-
-def save_model_to_config(model_name):
-    config = getConfig()
-    if 'model' not in config:
-        config['model'] = {}
-
-    config['model']['stable-diffusion'] = model_name
-
-    setConfig(config)
-
-@app.post('/image')
-def image(req : ImageRequest):
-    from sd_internal import runtime
-
-    r = Request()
-    r.session_id = req.session_id
-    r.prompt = req.prompt
-    r.negative_prompt = req.negative_prompt
-    r.init_image = req.init_image
-    r.mask = req.mask
-    r.num_outputs = req.num_outputs
-    r.num_inference_steps = req.num_inference_steps
-    r.guidance_scale = req.guidance_scale
-    r.width = req.width
-    r.height = req.height
-    r.seed = req.seed
-    r.prompt_strength = req.prompt_strength
-    r.sampler = req.sampler
-    # r.allow_nsfw = req.allow_nsfw
-    r.turbo = req.turbo
-    r.use_cpu = req.use_cpu
-    r.use_full_precision = req.use_full_precision
-    r.save_to_disk_path = req.save_to_disk_path
-    r.use_upscale: str = req.use_upscale
-    r.use_face_correction = req.use_face_correction
-    r.show_only_filtered_image = req.show_only_filtered_image
-    r.output_format = req.output_format
-
-    r.stream_progress_updates = True # the underlying implementation only supports streaming
-    r.stream_image_progress = req.stream_image_progress
-
-    r.use_stable_diffusion_model = resolve_model_to_use(req.use_stable_diffusion_model)
-
-    save_model_to_config(req.use_stable_diffusion_model)
-
-    try:
-        if not req.stream_progress_updates:
-            r.stream_image_progress = False
-
-        res = runtime.mk_img(r)
-
-        if req.stream_progress_updates:
-            return StreamingResponse(res, media_type='application/json')
-        else: # compatibility mode: buffer the streaming responses, and return the last one
-            last_result = None
-
-            for result in res:
-                last_result = result
-
-            return json.loads(last_result)
-    except Exception as e:
-        print(traceback.format_exc())
-        return HTTPException(status_code=500, detail=str(e))
-
-@app.get('/image/stop')
-def stop():
-    try:
-        if model_is_loading:
-            return {'ERROR'}
-
-        from sd_internal import runtime
-        runtime.stop_processing = True
-
-        return {'OK'}
-    except Exception as e:
-        print(traceback.format_exc())
-        return HTTPException(status_code=500, detail=str(e))
-
-@app.get('/image/tmp/{session_id}/{img_id}')
-def get_image(session_id, img_id):
-    from sd_internal import runtime
-    buf = runtime.temp_images[session_id + '/' + img_id]
-    buf.seek(0)
-    return StreamingResponse(buf, media_type='image/jpeg')
-
-@app.post('/app_config')
-async def setAppConfig(req : SetAppConfigRequest):
-    try:
-        config = {
-            'update_branch': req.update_branch
-        }
-
-        config_json_str = json.dumps(config)
-        config_bat_str = f'@set update_branch={req.update_branch}'
-        config_sh_str = f'export update_branch={req.update_branch}'
-
-        config_json_path = os.path.join(CONFIG_DIR, 'config.json')
-        config_bat_path = os.path.join(CONFIG_DIR, 'config.bat')
-        config_sh_path = os.path.join(CONFIG_DIR, 'config.sh')
-
-        with open(config_json_path, 'w') as f:
-            f.write(config_json_str)
-
-        with open(config_bat_path, 'w') as f:
-            f.write(config_bat_str)
-
-        with open(config_sh_path, 'w') as f:
-            f.write(config_sh_str)
-
-        return {'OK'}
-    except Exception as e:
-        print(traceback.format_exc())
-        return HTTPException(status_code=500, detail=str(e))
-
-@app.get('/app_config')
-def getAppConfig():
+def getConfig(default_val=APP_CONFIG_DEFAULTS):
     try:
         config_json_path = os.path.join(CONFIG_DIR, 'config.json')
-
         if not os.path.exists(config_json_path):
-            return HTTPException(status_code=500, detail="No config file")
-
-        with open(config_json_path, 'r') as f:
+            return default_val
+        with open(config_json_path, 'r', encoding='utf-8') as f:
             return json.load(f)
     except Exception as e:
+        print(str(e))
         print(traceback.format_exc())
-        return HTTPException(status_code=500, detail=str(e))
-
-def getConfig():
-    try:
-        config_json_path = os.path.join(CONFIG_DIR, 'config.json')
-
-        if not os.path.exists(config_json_path):
-            return {}
-
-        with open(config_json_path, 'r') as f:
-            return json.load(f)
-    except Exception as e:
-        return {}
+        return default_val
 
 def setConfig(config):
-    try:
+    try: # config.json
         config_json_path = os.path.join(CONFIG_DIR, 'config.json')
-
-        with open(config_json_path, 'w') as f:
-            return json.dump(config, f)
+        with open(config_json_path, 'w', encoding='utf-8') as f:
+            json.dump(config, f)
     except:
         print(traceback.format_exc())
 
-@app.get('/models')
+    try: # config.bat
+        config_bat_path = os.path.join(CONFIG_DIR, 'config.bat')
+        config_bat = []
+
+        if 'update_branch' in config:
+            config_bat.append(f"@set update_branch={config['update_branch']}")
+        if os.getenv('SD_UI_BIND_PORT') is not None:
+            config_bat.append(f"@set SD_UI_BIND_PORT={os.getenv('SD_UI_BIND_PORT')}")
+        if os.getenv('SD_UI_BIND_IP') is not None:
+            config_bat.append(f"@set SD_UI_BIND_IP={os.getenv('SD_UI_BIND_IP')}")
+
+        if len(config_bat) > 0:
+            with open(config_bat_path, 'w', encoding='utf-8') as f:
+                f.write('\r\n'.join(config_bat))
+    except:
+        print(traceback.format_exc())
+
+    try: # config.sh
+        config_sh_path = os.path.join(CONFIG_DIR, 'config.sh')
+        config_sh = ['#!/bin/bash']
+
+        if 'update_branch' in config:
+            config_sh.append(f"export update_branch={config['update_branch']}")
+        if os.getenv('SD_UI_BIND_PORT') is not None:
+            config_sh.append(f"export SD_UI_BIND_PORT={os.getenv('SD_UI_BIND_PORT')}")
+        if os.getenv('SD_UI_BIND_IP') is not None:
+            config_sh.append(f"export SD_UI_BIND_IP={os.getenv('SD_UI_BIND_IP')}")
+
+        if len(config_sh) > 1:
+            with open(config_sh_path, 'w', encoding='utf-8') as f:
+                f.write('\n'.join(config_sh))
+    except:
+        print(traceback.format_exc())
+
+def resolve_model_to_use(model_name:str, model_type:str, model_dir:str, model_extensions:list, default_models=[]):
+    model_dirs = [os.path.join(MODELS_DIR, model_dir), SD_DIR]
+    if not model_name: # When None try user configured model.
+        config = getConfig()
+        if 'model' in config and model_type in config['model']:
+            model_name = config['model'][model_type]
+    if model_name:
+        # Check models directory
+        models_dir_path = os.path.join(MODELS_DIR, model_dir, model_name)
+        for model_extension in model_extensions:
+            if os.path.exists(models_dir_path + model_extension):
+                return models_dir_path
+            if os.path.exists(model_name + model_extension):
+                # Direct Path to file
+                model_name = os.path.abspath(model_name)
+                return model_name
+    # Default locations
+    if model_name in default_models:
+        default_model_path = os.path.join(SD_DIR, model_name)
+        for model_extension in model_extensions:
+            if os.path.exists(default_model_path + model_extension):
+                return default_model_path
+    # Can't find requested model, check the default paths.
+    for default_model in default_models:
+        for model_dir in model_dirs:
+            default_model_path = os.path.join(model_dir, default_model)
+            for model_extension in model_extensions:
+                if os.path.exists(default_model_path + model_extension):
+                    if model_name is not None:
+                        print(f'Could not find the configured custom model {model_name}{model_extension}. Using the default one: {default_model_path}{model_extension}')
+                    return default_model_path
+    raise Exception('No valid models found.')
+
+def resolve_ckpt_to_use(model_name:str=None):
+    return resolve_model_to_use(model_name, model_type='stable-diffusion', model_dir='stable-diffusion', model_extensions=['.ckpt'], default_models=APP_CONFIG_DEFAULT_MODELS)
+
+def resolve_vae_to_use(model_name:str=None):
+    try:
+        return resolve_model_to_use(model_name, model_type='vae', model_dir='vae', model_extensions=['.vae.pt', '.ckpt'], default_models=[])
+    except:
+        return None
+
+class SetAppConfigRequest(BaseModel):
+    update_branch: str = None
+    render_devices: Union[List[str], List[int], str, int] = None
+    model_vae: str = None
+    ui_open_browser_on_start: bool = None
+
+@app.post('/app_config')
+async def setAppConfig(req : SetAppConfigRequest):
+    config = getConfig()
+    if req.update_branch is not None:
+        config['update_branch'] = req.update_branch
+    if req.render_devices is not None:
+        update_render_devices_in_config(config, req.render_devices)
+    if req.ui_open_browser_on_start is not None:
+        if 'ui' not in config:
+            config['ui'] = {}
+        config['ui']['open_browser_on_start'] = req.ui_open_browser_on_start
+    try:
+        setConfig(config)
+
+        if req.render_devices:
+            update_render_threads()
+
+        return JSONResponse({'status': 'OK'}, headers=NOCACHE_HEADERS)
+    except Exception as e:
+        print(traceback.format_exc())
+        raise HTTPException(status_code=500, detail=str(e))
+
 def getModels():
     models = {
         'active': {
             'stable-diffusion': 'sd-v1-4',
+            'vae': '',
         },
         'options': {
             'stable-diffusion': ['sd-v1-4'],
+            'vae': [],
         },
     }
 
+    def listModels(models_dirname, model_type, model_extensions):
+        models_dir = os.path.join(MODELS_DIR, models_dirname)
+        if not os.path.exists(models_dir):
+            os.makedirs(models_dir)
+
+        for file in os.listdir(models_dir):
+            for model_extension in model_extensions:
+                if file.endswith(model_extension):
+                    model_name = file[:-len(model_extension)]
+                    models['options'][model_type].append(model_name)
+
+        models['options'][model_type] = [*set(models['options'][model_type])] # remove duplicates
+        models['options'][model_type].sort()
+
     # custom models
-    sd_models_dir = os.path.join(MODELS_DIR, 'stable-diffusion')
-    for file in os.listdir(sd_models_dir):
-        if file.endswith('.ckpt'):
-            model_name = os.path.splitext(file)[0]
-            models['options']['stable-diffusion'].append(model_name)
+    listModels(models_dirname='stable-diffusion', model_type='stable-diffusion', model_extensions=['.ckpt'])
+    listModels(models_dirname='vae', model_type='vae', model_extensions=['.vae.pt', '.ckpt'])
 
     # legacy
     custom_weight_path = os.path.join(SD_DIR, 'custom-model.ckpt')
     if os.path.exists(custom_weight_path):
-        models['active']['stable-diffusion'] = 'custom-model'
         models['options']['stable-diffusion'].append('custom-model')
 
-    config = getConfig()
-    if 'model' in config and 'stable-diffusion' in config['model']:
-        models['active']['stable-diffusion'] = config['model']['stable-diffusion']
-
     return models
 
-@app.get('/modifiers.json')
-def read_modifiers():
-    headers = {"Cache-Control": "no-cache, no-store, must-revalidate", "Pragma": "no-cache", "Expires": "0"}
-    return FileResponse(os.path.join(SD_UI_DIR, 'modifiers.json'), headers=headers)
+def getUIPlugins():
+    plugins = []
 
-@app.get('/output_dir')
-def read_home_dir():
-    return {outpath}
+    for file in os.listdir(UI_PLUGINS_DIR):
+        if file.endswith('.plugin.js'):
+            plugins.append(f'/plugins/{file}')
+
+    return plugins
+
+@app.get('/get/{key:path}')
+def read_web_data(key:str=None):
+    if not key: # /get without parameters, stable-diffusion easter egg.
+        raise HTTPException(status_code=418, detail="StableDiffusion is drawing a teapot!") # HTTP418 I'm a teapot
+    elif key == 'app_config':
+        config = getConfig(default_val=None)
+        if config is None:
+            config = APP_CONFIG_DEFAULTS
+        return JSONResponse(config, headers=NOCACHE_HEADERS)
+    elif key == 'devices':
+        config = getConfig()
+        devices = task_manager.get_devices()
+        devices['config'] = config.get('render_devices', "auto")
+        return JSONResponse(devices, headers=NOCACHE_HEADERS)
+    elif key == 'models':
+        return JSONResponse(getModels(), headers=NOCACHE_HEADERS)
+    elif key == 'modifiers': return FileResponse(os.path.join(SD_UI_DIR, 'modifiers.json'), headers=NOCACHE_HEADERS)
+    elif key == 'output_dir': return JSONResponse({ 'output_dir': outpath }, headers=NOCACHE_HEADERS)
+    elif key == 'ui_plugins': return JSONResponse(getUIPlugins(), headers=NOCACHE_HEADERS)
+    else:
+        raise HTTPException(status_code=404, detail=f'Request for unknown {key}') # HTTP404 Not Found
+
+@app.get('/ping') # Get server and optionally session status.
+def ping(session_id:str=None):
+    if task_manager.is_alive() <= 0: # Check that render threads are alive.
+        if task_manager.current_state_error: raise HTTPException(status_code=500, detail=str(task_manager.current_state_error))
+        raise HTTPException(status_code=500, detail='Render thread is dead.')
+    if task_manager.current_state_error and not isinstance(task_manager.current_state_error, StopAsyncIteration): raise HTTPException(status_code=500, detail=str(task_manager.current_state_error))
+    # Alive
+    response = {'status': str(task_manager.current_state)}
+    if session_id:
+        task = task_manager.get_cached_task(session_id, update_ttl=True)
+        if task:
+            response['task'] = id(task)
+            if task.lock.locked():
+                response['session'] = 'running'
+            elif isinstance(task.error, StopAsyncIteration):
+                response['session'] = 'stopped'
+            elif task.error:
+                response['session'] = 'error'
+            elif not task.buffer_queue.empty():
+                response['session'] = 'buffer'
+            elif task.response:
+                response['session'] = 'completed'
+            else:
+                response['session'] = 'pending'
+    response['devices'] = task_manager.get_devices()
+    return JSONResponse(response, headers=NOCACHE_HEADERS)
+
+def save_model_to_config(ckpt_model_name, vae_model_name):
+    config = getConfig()
+    if 'model' not in config:
+        config['model'] = {}
+
+    config['model']['stable-diffusion'] = ckpt_model_name
+    config['model']['vae'] = vae_model_name
+
+    if vae_model_name is None or vae_model_name == "":
+        del config['model']['vae']
+
+    setConfig(config)
+
+def update_render_devices_in_config(config, render_devices):
+    if render_devices not in ('cpu', 'auto') and not render_devices.startswith('cuda:'):
+        raise HTTPException(status_code=400, detail=f'Invalid render device requested: {render_devices}')
+
+    if render_devices.startswith('cuda:'):
+        render_devices = render_devices.split(',')
+
+    config['render_devices'] = render_devices
+
+@app.post('/render')
+def render(req : task_manager.ImageRequest):
+    try:
+        save_model_to_config(req.use_stable_diffusion_model, req.use_vae_model)
+        req.use_stable_diffusion_model = resolve_ckpt_to_use(req.use_stable_diffusion_model)
+        req.use_vae_model = resolve_vae_to_use(req.use_vae_model)
+        new_task = task_manager.render(req)
+        response = {
+            'status': str(task_manager.current_state), 
+            'queue': len(task_manager.tasks_queue),
+            'stream': f'/image/stream/{req.session_id}/{id(new_task)}',
+            'task': id(new_task)
+        }
+        return JSONResponse(response, headers=NOCACHE_HEADERS)
+    except ChildProcessError as e: # Render thread is dead
+        raise HTTPException(status_code=500, detail=f'Rendering thread has died.') # HTTP500 Internal Server Error
+    except ConnectionRefusedError as e: # Unstarted task pending, deny queueing more than one.
+        raise HTTPException(status_code=503, detail=f'Session {req.session_id} has an already pending task.') # HTTP503 Service Unavailable
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+@app.get('/image/stream/{session_id:str}/{task_id:int}')
+def stream(session_id:str, task_id:int):
+    #TODO Move to WebSockets ??
+    task = task_manager.get_cached_task(session_id, update_ttl=True)
+    if not task: raise HTTPException(status_code=410, detail='No request received.') # HTTP410 Gone
+    if (id(task) != task_id): raise HTTPException(status_code=409, detail=f'Wrong task id received. Expected:{id(task)}, Received:{task_id}') # HTTP409 Conflict
+    if task.buffer_queue.empty() and not task.lock.locked():
+        if task.response:
+            #print(f'Session {session_id} sending cached response')
+            return JSONResponse(task.response, headers=NOCACHE_HEADERS)
+        raise HTTPException(status_code=425, detail='Too Early, task not started yet.') # HTTP425 Too Early
+    #print(f'Session {session_id} opened live render stream {id(task.buffer_queue)}')
+    return StreamingResponse(task.read_buffer_generator(), media_type='application/json')
+
+@app.get('/image/stop')
+def stop(session_id:str=None):
+    if not session_id:
+        if task_manager.current_state == task_manager.ServerStates.Online or task_manager.current_state == task_manager.ServerStates.Unavailable:
+            raise HTTPException(status_code=409, detail='Not currently running any tasks.') # HTTP409 Conflict
+        task_manager.current_state_error = StopAsyncIteration('')
+        return {'OK'}
+    task = task_manager.get_cached_task(session_id, update_ttl=False)
+    if not task: raise HTTPException(status_code=404, detail=f'Session {session_id} has no active task.') # HTTP404 Not Found
+    if isinstance(task.error, StopAsyncIteration): raise HTTPException(status_code=409, detail=f'Session {session_id} task is already stopped.') # HTTP409 Conflict
+    task.error = StopAsyncIteration('')
+    return {'OK'}
+
+@app.get('/image/tmp/{session_id}/{img_id:int}')
+def get_image(session_id, img_id):
+    task = task_manager.get_cached_task(session_id, update_ttl=True)
+    if not task: raise HTTPException(status_code=410, detail=f'Session {session_id} has not submitted a task.') # HTTP410 Gone
+    if not task.temp_images[img_id]: raise HTTPException(status_code=425, detail='Too Early, task data is not available yet.') # HTTP425 Too Early
+    try:
+        img_data = task.temp_images[img_id]
+        img_data.seek(0)
+        return StreamingResponse(img_data, media_type='image/jpeg')
+    except KeyError as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+@app.get('/')
+def read_root():
+    return FileResponse(os.path.join(SD_UI_DIR, 'index.html'), headers=NOCACHE_HEADERS)
+
+@app.on_event("shutdown")
+def shutdown_event(): # Signal render thread to close on shutdown
+    task_manager.current_state_error = SystemExit('Application shutting down.')
 
 # don't log certain requests
 class LogSuppressFilter(logging.Filter):
@@ -323,10 +375,28 @@ class LogSuppressFilter(logging.Filter):
         for prefix in ACCESS_LOG_SUPPRESS_PATH_PREFIXES:
             if path.find(prefix) != -1:
                 return False
-
         return True
-
 logging.getLogger('uvicorn.access').addFilter(LogSuppressFilter())
 
+# Start the task_manager
+task_manager.default_model_to_load = resolve_ckpt_to_use()
+task_manager.default_vae_to_load = resolve_vae_to_use()
+
+def update_render_threads():
+    config = getConfig()
+    render_devices = config.get('render_devices', 'auto')
+    active_devices = task_manager.get_devices()['active'].keys()
+
+    print('requesting for render_devices', render_devices)
+    task_manager.update_render_threads(render_devices, active_devices)
+
+update_render_threads()
+
 # start the browser ui
-import webbrowser; webbrowser.open('http://localhost:9000')
+def open_browser():
+    config = getConfig()
+    ui = config.get('ui', {})
+    if ui.get('open_browser_on_start', True):
+        import webbrowser; webbrowser.open('http://localhost:9000')
+
+open_browser()