Include C:\Windows\System32 in the path anyway, to avoid the errors some users have

Live Preview, In-Painting Editor, New Sampler Options, VRAM (Memory) Usage Improvements, Progress Bar, Re-organized UI with more space for images, Update to the latest fork version

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+ko_fi: cmdr2_stablediffusion_ui

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: 
+ - Browser:
+ - Version:
+
+**Smartphone (please complete the following information):**
+ - Device: 
+ - OS: 
+ - Browser 
+ - Version 
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -1,4 +1,5 @@
 __pycache__
 installer
 installer.tar
-dist
+dist
+.idea/*

CONTRIBUTING.md

@@ -0,0 +1,55 @@
+Hi there, these instructions are meant for the developers of this project.
+
+If you only want to use the Stable Diffusion UI, you've downloaded the wrong file. In that case, please download and follow the instructions at https://github.com/cmdr2/stable-diffusion-ui#installation
+
+Thanks
+
+# For developers:
+
+If you would like to contribute to this project, there is a discord for dicussion:
+[![Discord Server](https://badgen.net/badge/icon/discord?icon=discord&label)](https://discord.com/invite/u9yhsFmEkB)
+
+## Development environment for UI (frontend and server) changes
+This is in-flux, but one way to get a development environment running for editing the UI of this project is:
+(swap `.sh` or `.bat` in instructions depending on your environment, and be sure to adjust any paths to match where you're working)
+
+1) `git clone` the repository, e.g. to `/projects/stable-diffusion-ui-repo`
+2) Download the pre-built end user archive from the link on github, and extract it, e.g. to `/projects/stable-diffusion-ui-archive`
+3) `cd /projects/stable-diffusion-ui-archive` and run the script to set up and start the project, e.g. `start.sh`
+4) Check you can view and generate images on `localhost:9000`
+5) Close the server, and edit `/projects/stable-diffusion-ui-archive/scripts/on_env_start.sh`
+6) Comment out the lines near the bottom that copies the `files/ui` folder, e.g:
+  
+  for `.sh`
+```
+# rm -rf ui
+# cp -Rf sd-ui-files/ui .
+# cp sd-ui-files/scripts/on_sd_start.sh scripts/
+# cp sd-ui-files/scripts/start.sh .
+``` 
+for `.bat`
+```
+REM @xcopy sd-ui-files\ui ui /s /i /Y
+REM @copy sd-ui-files\scripts\on_sd_start.bat scripts\ /Y
+REM @copy "sd-ui-files\scripts\Start Stable Diffusion UI.cmd" . /Y
+``` 
+7) Comment out the line at the top of `/projects/stable-diffusion-ui-archive/scripts/on_sd_start.sh` that copies `on_env_start`. For e.g. `@copy sd-ui-files\scripts\on_env_start.bat scripts\ /Y`
+8) Delete the current `ui` folder at `/projects/stable-diffusion-ui-archive/ui`
+9) Now make a symlink between the repository clone (where you will be making changes) and this archive (where you will be running stable diffusion):
+`ln -s /projects/stable-diffusion-ui-repo/ui /projects/stable-diffusion-ui-archive/ui`
+or for Windows
+`mklink /D \projects\stable-diffusion-ui-archive\ui \projects\stable-diffusion-ui-repo\ui` (link name first, source repo dir second)
+9) Run the archive again `start.sh` and ensure you can still use the UI.
+10) Congrats, now any changes you make in your repo `ui` folder are linked to this running archive of the app and can be previewed in the browser.
+
+Check the `ui/frontend/build/README.md` for instructions on running and building the React code.
+
+## Development environment for Installer changes
+Build the Windows installer using Windows, and the Linux installer using Linux. Don't mix the two, and don't use WSL. An Ubuntu VM is fine for building the Linux installer on a Windows host.
+
+1. Install Miniconda 3 or Anaconda.
+2. Install `conda install -c conda-forge -y conda-pack`
+3. Open the Anaconda Prompt. Do not use WSL if you're building for Windows.
+4. Run `build.bat` or `./build.sh` depending on whether you're in Windows or Linux.
+5. Compress the `stable-diffusion-ui` folder created inside the `dist` folder. Make a `zip` for Windows, and `tar.xz` for Linux (smaller files, and Linux users already have tar).
+6. Make a new GitHub release and upload the Windows and Linux installer builds.

How to install and run.txt

@@ -1,6 +1,6 @@
 Congrats on downloading Stable Diffusion UI, version 2!
 
-If you haven't downloaded Stable Diffusion UI yet, please download from https://github.com/cmdr2/stable-diffusion-ui
+If you haven't downloaded Stable Diffusion UI yet, please download from https://github.com/cmdr2/stable-diffusion-ui#installation
 
 After downloading, to install please follow these instructions:
 

README BEFORE YOU RUN THIS.txt

@@ -0,0 +1,8 @@
+Hi there,
+
+What you have downloaded is meant for the developers of this project, not for users.
+
+If you only want to use the Stable Diffusion UI, you've downloaded the wrong file.
+Please download and follow the instructions at https://github.com/cmdr2/stable-diffusion-ui#installation
+
+Thanks

README.md

@@ -1,16 +1,30 @@
-# Stable Diffusion UI - v2 (beta)
-### A simple way to install and use [Stable Diffusion](https://github.com/CompVis/stable-diffusion) on your own computer (Win 10/11, Linux). No dependencies or technical knowledge required.
+# Stable Diffusion UI v2
+### A simple 1-click way to install and use [Stable Diffusion](https://github.com/CompVis/stable-diffusion) on your own computer. No dependencies or technical knowledge required.
 
-[![Discord Server](https://badgen.net/badge/icon/discord?icon=discord&label)](https://discord.com/invite/u9yhsFmEkB) (for support, and development discussion)
+<p float="left">
+  <a href="#installation"><img src="https://github.com/cmdr2/stable-diffusion-ui/raw/develop/media/download-win.png" width="200" /></a>
+  <a href="#installation"><img src="https://github.com/cmdr2/stable-diffusion-ui/raw/develop/media/download-linux.png" width="200" /></a>
+</p>
+
+[![Discord Server](https://badgen.net/badge/icon/discord?icon=discord&label)](https://discord.com/invite/u9yhsFmEkB) (for support, and development discussion) | [Troubleshooting guide for common problems](Troubleshooting.md)
+
+️‍🔥🎉 **New!** Live Preview, More Samplers, In-Painting, Face Correction (GFPGAN) and Upscaling (RealESRGAN) have been added!
+
+This distribution currently uses Stable Diffusion 1.4. Once the model for 1.5 becomes publicly available, the model in this distribution will be updated.
 
 # Features in the new v2 Version:
 - **No Dependencies or Technical Knowledge Required**: 1-click install for Windows 10/11 and Linux. *No dependencies*, no need for WSL or Docker or Conda or technical setup. Just download and run!
+- **Face Correction (GFPGAN) and Upscaling (RealESRGAN)**
+- **In-Painting**
+- **Live Preview**: See the image as the AI is drawing it
+- **Lots of Samplers**
 - **Image Modifiers**: A library of *modifier tags* like *"Realistic"*, *"Pencil Sketch"*, *"ArtStation"* etc. Experiment with various styles quickly.
 - **New UI**: with cleaner design
 - Supports "*Text to Image*" and "*Image to Image*"
 - **NSFW Setting**: A setting in the UI to control *NSFW content*
 - **Use CPU setting**: If you don't have a compatible graphics card, but still want to run it on your CPU.
 - **Auto-updater**: Gets you the latest improvements and bug-fixes to a rapidly evolving project.
+- **Low Memory Usage**: Creates 512x512 images with less than 4GB of VRAM!
 
 ![Screenshot](media/shot-v8.jpg?raw=true)
 
@@ -21,21 +35,23 @@
 You do not need anything else. You do not need WSL, Docker or Conda. The installer will take care of it.
 
 # Installation
-1. Download [for Windows](https://drive.google.com/file/d/1MY5gzsQHV_KREbYs3gw33QL4gGIlQRqj/view?usp=sharing) or [for Linux](https://drive.google.com/file/d/1Gwz1LVQUCart8HhCjrmXkS6TWKbTsLsR/view?usp=sharing) (this will be hosted on GitHub in the future).
+1. **Download** [for Windows](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.05/stable-diffusion-ui-win64.zip) or [for Linux](https://github.com/cmdr2/stable-diffusion-ui/releases/download/v2.05/stable-diffusion-ui-linux.tar.xz).
 
-2. Extract:
-  - For Windows: After unzipping the file, please move the `stable-diffusion-ui` folder to your `C:` (or any drive like D: at the top root level). For e.g. `C:\stable-diffusion-ui`. This will avoid a common problem with Windows (of file path length limits).
+2. **Extract**:
+  - For Windows: After unzipping the file, please move the `stable-diffusion-ui` folder to your `C:` (or any drive like D:, at the top root level), e.g. `C:\stable-diffusion-ui`. This will avoid a common problem with Windows (file path length limits).
   - For Linux: After extracting the .tar.xz file, please open a terminal, and go to the `stable-diffusion-ui` directory.
 
-3. Run:
+3. **Run**:
   - For Windows: `Start Stable Diffusion UI.cmd` by double-clicking it.
   - For Linux: In the terminal, run `./start.sh` (or `bash start.sh`)
 
 This will automatically install Stable Diffusion, set it up, and start the interface. No additional steps are needed.
 
+**To Uninstall:** Just delete the `stable-diffusion-ui` folder to uninstall all the downloaded packages.
+
 
 # Usage
-Open http://localhost:9000 in your browser (after running step 3 previously).
+Open http://localhost:9000 in your browser (after running step 3 previously). It may take a few moments for the back-end to be ready.
 
 ## With a text description
 1. Enter a text prompt, like `a photograph of an astronaut riding a horse` in the textbox.
@@ -47,22 +63,21 @@ Open http://localhost:9000 in your browser (after running step 3 previously).
 2. An optional text prompt can help you further describe the kind of image you want to generate.
 3. Press `Make Image`. See the image generated using your prompt.
 
+You can use Face Correction or Upscaling to improve the image further.
+
 **Pro tip:** You can also click `Use as Input` on a generated image, to use it as the input image for your next generation. This can be useful for sequentially refining the generated image with a single click.
 
 **Another tip:** Images with the same aspect ratio of your generated image work best. E.g. 1:1 if you're generating images sized 512x512.
 
-## Problems?
-Please ask on the new [discord server](https://discord.com/invite/u9yhsFmEkB), or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues) if this did not work for you (after trying the common [troubleshooting](#troubleshooting) steps)!
+## Problems? Troubleshooting
+Please try the common [troubleshooting](Troubleshooting.md) steps. If that doesn't fix it, please ask on the [discord server](https://discord.com/invite/u9yhsFmEkB), or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
 
 # Advanced Settings
 You can also set the configuration like `seed`, `width`, `height`, `num_outputs`, `num_inference_steps` and `guidance_scale` using the 'show' button next to 'Advanced settings'.
 
 Use the same `seed` number to get the same image for a certain prompt. This is useful for refining a prompt without losing the basic image design. Enable the `random images` checkbox to get random images.
 
-![Screenshot of advanced settings](media/config-v5.jpg?raw=true)
-
-# Troubleshooting
-The [Troubleshooting wiki page](https://github.com/cmdr2/stable-diffusion-ui/wiki/Troubleshooting) contains some common errors and their solutions. Please check that, and if it doesn't work, feel free to ask on the [discord server](https://discord.com/invite/u9yhsFmEkB) or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
+![Screenshot of advanced settings](media/config-v6.png?raw=true)
 
 # What is this? Why no Docker?
 This version is a 1-click installer. You don't need WSL or Docker or anything beyond a working NVIDIA GPU with an updated driver. You don't need to use the command-line at all. Even if you don't have a compatible GPU, you can run it on your CPU (albeit very slowly).
@@ -79,4 +94,4 @@ Also, please feel free to submit a pull request, if you have any code contributi
 # Disclaimer
 The authors of this project are not responsible for any content generated using this interface.
 
-This license of this software forbids you from sharing any content that violates any laws, produce any harm to a person, disseminate any personal information that would be meant for harm, spread misinformation and target vulnerable groups. For the full list of restrictions please read [the license](LICENSE).
+The license of this software forbids you from sharing any content that violates any laws, produce any harm to a person, disseminate any personal information that would be meant for harm, spread misinformation, or target vulnerable groups. For the full list of restrictions please read [the license](LICENSE). You agree to these terms by using this software.

Troubleshooting.md

@@ -0,0 +1,46 @@
+Common issues and their solutions. If these solutions don't work, please feel free to ask at the [discord server](https://discord.com/invite/u9yhsFmEkB) or [file an issue](https://github.com/cmdr2/stable-diffusion-ui/issues).
+
+## RuntimeError: CUDA out of memory
+This can happen if your PC has less than 6GB of VRAM.
+
+Try disabling the "Turbo mode" setting under "Advanced Settings", since that takes an additional 1 GB of VRAM (to increase the speed).
+
+Additionally, a common reason for this error is that you're using an initial image larger than 768x768 pixels. Try using a smaller initial image.
+
+Also try generating smaller sized images.
+
+## No ldm found, or antlr4 or any other missing module, or ClobberError: This transaction has incompatible packages due to a shared path
+On Windows, please ensure that you had placed the `stable-diffusion-ui` folder after unzipping to the root of C: or D: (or any drive). For e.g. `C:\stable-diffusion-ui`. **Note:** This has to be done **before** you start the installation process. If you have already installed (and are facing this error), please delete the installed folder, and start fresh by unzipping and placing the folder at the top of your drive.
+
+This error can also be caused if you already have conda/miniconda/anaconda installed, due to package conflicts. Please open your Anaconda Prompt, and run `conda clean --all` to clean up unused packages.
+
+If nothing works, this could be due to a corrupted installation. Please try reinstalling this, by deleting the installed folder, and unzipping from the downloaded zip file.
+
+## Killed uvicorn server:app --app-dir ... --port 9000 --host 0.0.0.0
+This happens if your PC ran out of RAM. Stable Diffusion requires a lot of RAM, and requires atleast 10 GB of RAM to work well. You can also try closing all other applications before running Stable Diffusion UI.
+
+## Green image generated
+This usually happens if you're running NVIDIA 1650 or 1660 Super. To solve this, please close and run the Stable Diffusion command on your computer. If you're using the older Docker-based solution (v1), please upgrade to v2: https://github.com/cmdr2/stable-diffusion-ui/tree/v2#installation
+
+If you're still seeing this error, please try enabling "Full Precision" under "Advanced Settings" in the Stable Diffusion UI.
+
+## './docker-compose.yml' is invalid:
+> ERROR: The Compose file './docker-compose.yml' is invalid because:
+> services.stability-ai.deploy.resources.reservations value Additional properties are not allowed ('devices' was unexpected)
+
+Please ensure you have `docker-compose` version 1.29 or higher. Check `docker-compose --version`, and if required [update it to 1.29](https://docs.docker.com/compose/install/). (Thanks [HVRyan](https://github.com/HVRyan))
+
+## RuntimeError: Found no NVIDIA driver on your system:
+If you have an NVIDIA GPU and the latest [NVIDIA driver](http://www.nvidia.com/Download/index.aspx), please ensure that you've installed [nvidia-container-toolkit](https://stackoverflow.com/a/58432877). (Thanks [u/exintrovert420](https://www.reddit.com/user/exintrovert420/))
+
+## Some other process is already running at port 9000 / port 9000 could not be bound
+You can override the port used. Please change `docker-compose.yml` inside the project directory, and update the line `9000:9000` to `1337:9000` (where 1337 is whichever port number you want).
+
+After doing this, please restart your server, by running `./server restart`.
+
+After this, you can access the server at `http://localhost:1337` (where 1337 is the new port you specified earlier).
+
+## RuntimeError: CUDA error: unknown error
+Please ensure that you have an NVIDIA GPU and the latest [NVIDIA driver](http://www.nvidia.com/Download/index.aspx), and that you've installed [nvidia-container-toolkit](https://stackoverflow.com/a/58432877).
+
+Also, if you are using WSL (Windows), please ensure you have the latest WSL kernel by running `wsl --shutdown` and then `wsl --update`. (Thanks [AndrWeisR](https://github.com/AndrWeisR))

build.bat

@@ -1,3 +1,13 @@
+@echo off
+
+@echo "Hi there, what you are running is meant for the developers of this project, not for users." & echo.
+@echo "If you only want to use the Stable Diffusion UI, you've downloaded the wrong file."
+@echo "Please download and follow the instructions at https://github.com/cmdr2/stable-diffusion-ui#installation" & echo.
+@echo "If you are actually a developer of this project, please type Y and press enter" & echo.
+
+set /p answer=Are you a developer of this project (Y/N)?
+if /i "%answer:~,1%" NEQ "Y" exit /b
+
 @mkdir dist\stable-diffusion-ui
 
 @echo "Downloading components for the installer.."

build.sh

@@ -1,5 +1,16 @@
 #!/bin/bash
 
+printf "Hi there, what you are running is meant for the developers of this project, not for users.\n\n"
+printf "If you only want to use the Stable Diffusion UI, you've downloaded the wrong file.\n"
+printf "Please download and follow the instructions at https://github.com/cmdr2/stable-diffusion-ui#installation\n\n"
+printf "If you are actually a developer of this project, please type Y and press enter\n\n"
+
+read -p "Are you a developer of this project (Y/N) " yn
+case $yn in
+    [Yy]* ) ;;
+    * ) exit;;
+esac
+
 mkdir -p dist/stable-diffusion-ui
 
 echo "Downloading components for the installer.."
@@ -23,11 +34,13 @@ tar -xf ../../installer.tar -C installer
 mkdir scripts
 
 cp ../../scripts/on_env_start.sh scripts/
-cp "../../scripts/start.sh" .
+cp ../../scripts/start.sh .
 cp ../../LICENSE .
 cp "../../CreativeML Open RAIL-M License" .
 cp "../../How to install and run.txt" .
 
+chmod u+x start.sh
+
 echo "Build ready. Zip the 'dist/stable-diffusion-ui' folder."
 
 echo "Cleaning up.."

scripts/on_env_start.bat

@@ -1,30 +1,61 @@
-@echo. & echo "Stable Diffusion UI" & echo.
+@echo off
+
+@echo. & echo "Stable Diffusion UI - v2" & echo.
+
+set PATH=C:\Windows\System32;%PATH%
 
 @cd ..
 
+if exist "scripts\config.bat" (
+    @call scripts\config.bat
+)
+
+if "%update_branch%"=="" (
+    set update_branch=main
+)
+
+@>nul grep -c "conda_sd_ui_deps_installed" scripts\install_status.txt
+@if "%ERRORLEVEL%" NEQ "0" (
+    for /f "tokens=*" %%a in ('python -c "import os; parts = os.getcwd().split(os.path.sep); print(len(parts))"') do if "%%a" NEQ "2" (
+        echo. & echo "!!!! WARNING !!!!" & echo.
+        echo "Your 'stable-diffusion-ui' folder is at %cd%" & echo.
+        echo "The 'stable-diffusion-ui' folder needs to be at the top of your drive, for e.g. 'C:\stable-diffusion-ui' or 'D:\stable-diffusion-ui' etc."
+        echo "Not placing this folder at the top of a drive can cause errors on some computers."
+        echo. & echo "Recommended: Please close this window and move the 'stable-diffusion-ui' folder to the top of a drive. For e.g. 'C:\stable-diffusion-ui'. Then run the installer again." & echo.
+        echo "Not Recommended: If you're sure that you want to install at the current location, please press any key to continue." & echo.
+
+        pause
+    )
+)
+
 @>nul grep -c "sd_ui_git_cloned" scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
-    @echo "Stable Diffusion UI's git repository was already installed. Updating.."
+    @echo "Stable Diffusion UI's git repository was already installed. Updating from %update_branch%.."
 
     @cd sd-ui-files
 
     @call git reset --hard
+    @call git checkout "%update_branch%"
     @call git pull
 
     @cd ..
 ) else (
     @echo. & echo "Downloading Stable Diffusion UI.." & echo.
+    @echo "Using the %update_branch% channel" & echo.
 
-    @call git clone https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files && (
+    @call git clone -b "%update_branch%" https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files && (
         @echo sd_ui_git_cloned >> scripts\install_status.txt
     ) || (
-        @echo "Error downloading Stable Diffusion UI. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues"
+        @echo "Error downloading Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         @exit /b
     )
 )
 
 @xcopy sd-ui-files\ui ui /s /i /Y
-@xcopy sd-ui-files\scripts scripts /s /i /Y
+@copy sd-ui-files\scripts\on_sd_start.bat scripts\ /Y
+@copy "sd-ui-files\scripts\Start Stable Diffusion UI.cmd" . /Y
 
 @call scripts\on_sd_start.bat
+
+@pause

scripts/on_env_start.sh

@@ -1,28 +1,43 @@
+#!/bin/bash
+
 printf "\n\nStable Diffusion UI\n\n"
 
+if [ -f "scripts/config.sh" ]; then
+    source scripts/config.sh
+fi
+
+if [ "$update_branch" == "" ]; then
+    export update_branch="main"
+fi
+
 if [ -f "scripts/install_status.txt" ] && [ `grep -c sd_ui_git_cloned scripts/install_status.txt` -gt "0" ]; then
-    echo "Stable Diffusion UI's git repository was already installed. Updating.."
+    echo "Stable Diffusion UI's git repository was already installed. Updating from $update_branch.."
 
     cd sd-ui-files
 
     git reset --hard
+    git checkout "$update_branch"
     git pull
 
     cd ..
 else
     printf "\n\nDownloading Stable Diffusion UI..\n\n"
+    printf "Using the $update_branch channel\n\n"
 
-    if git clone https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files ; then
+    if git clone -b "$update_branch" https://github.com/cmdr2/stable-diffusion-ui.git sd-ui-files ; then
         echo sd_ui_git_cloned >> scripts/install_status.txt
     else
-        printf "\n\nError downloading Stable Diffusion UI. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\n\n"
+        printf "\n\nError downloading Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
 fi
 
-cp -Rf sd-ui-files/ui ui
-cp -Rf sd-ui-files/scripts/* scripts/
-cp "scripts/start.sh" .
+rm -rf ui
+cp -Rf sd-ui-files/ui .
+cp sd-ui-files/scripts/on_sd_start.sh scripts/
+cp sd-ui-files/scripts/start.sh .
 
 ./scripts/on_sd_start.sh
+
+read -p "Press any key to continue"

scripts/on_sd_start.bat

@@ -1,4 +1,11 @@
-@set cmd_had_error=F
+@echo off
+
+@copy sd-ui-files\scripts\on_env_start.bat scripts\ /Y
+
+@REM Caution, this file will make your eyes and brain bleed. It's such an unholy mess.
+@REM Note to self: Please rewrite this in Python. For the sake of your own sanity.
+
+@call python -c "import os; import shutil; frm = 'sd-ui-files\\ui\\hotfix\\9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'; dst = os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'transformers', '9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'); shutil.copyfile(frm, dst) if os.path.exists(dst) else print(''); print('Hotfixed broken JSON file from OpenAI');"
 
 @>nul grep -c "sd_git_cloned" scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
@@ -8,6 +15,10 @@
 
     @call git reset --hard
     @call git pull
+    @call git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+
+    @call git apply ..\ui\sd_internal\ddim_callback.patch
+    @call git apply ..\ui\sd_internal\env_yaml.patch
 
     @cd ..
 ) else (
@@ -16,18 +27,18 @@
     @call git clone https://github.com/basujindal/stable-diffusion.git && (
         @echo sd_git_cloned >> scripts\install_status.txt
     ) || (
-        @set cmd_had_error=T
-    )
-
-    if "%ERRORLEVEL%" NEQ "0" (
-        @set cmd_had_error=T
-    )
-
-    if "%cmd_had_error%"=="T" (
-        @echo "Error downloading Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues"
+        @echo "Error downloading Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         @exit /b
     )
+
+    @cd stable-diffusion
+    @call git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+
+    @call git apply ..\ui\sd_internal\ddim_callback.patch
+    @call git apply ..\ui\sd_internal\env_yaml.patch
+
+    @cd ..
 )
 
 @cd stable-diffusion
@@ -35,25 +46,92 @@
 @>nul grep -c "conda_sd_env_created" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
     @echo "Packages necessary for Stable Diffusion were already installed"
+
+    @call conda activate .\env
 ) else (
     @echo. & echo "Downloading packages necessary for Stable Diffusion.." & echo. & echo "***** This will take some time (depending on the speed of the Internet connection) and may appear to be stuck, but please be patient ***** .." & echo.
 
     @rmdir /s /q .\env
 
-    @call conda env create --prefix env -f environment.yaml && (
-        @echo conda_sd_env_created >> ..\scripts\install_status.txt
-    ) || (
-        @set cmd_had_error=T
-    )
+    @REM prevent conda from using packages from the user's home directory, to avoid conflicts
+    @set PYTHONNOUSERSITE=1
 
-    if "%cmd_had_error%"=="T" (
-        echo "Error installing the packages necessary for Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues"
+    @call conda env create --prefix env -f environment.yaml || (
+        @echo. & echo "Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
+
+    @call conda activate .\env
+
+    @call conda install -c conda-forge -y --prefix env antlr4-python3-runtime=4.8 || (
+        @echo. & echo "Error installing antlr4-python3-runtime for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    for /f "tokens=*" %%a in ('python -c "import torch; import ldm; import transformers; import numpy; import antlr4; print(42)"') do if "%%a" NEQ "42" (
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    @echo conda_sd_env_created >> ..\scripts\install_status.txt
 )
 
-@call conda activate .\env
+set PATH=C:\Windows\System32;%PATH%
+
+@>nul grep -c "conda_sd_gfpgan_deps_installed" ..\scripts\install_status.txt
+@if "%ERRORLEVEL%" EQU "0" (
+    @echo "Packages necessary for GFPGAN (Face Correction) were already installed"
+) else (
+    @echo. & echo "Downloading packages necessary for GFPGAN (Face Correction).." & echo.
+
+    @set PYTHONNOUSERSITE=1
+
+    @call pip install -e git+https://github.com/TencentARC/GFPGAN#egg=GFPGAN || (
+        @echo. & echo "Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    @call pip install basicsr==1.4.2 || (
+        @echo. & echo "Error installing the basicsr package necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    for /f "tokens=*" %%a in ('python -c "from gfpgan import GFPGANer; print(42)"') do if "%%a" NEQ "42" (
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    @echo conda_sd_gfpgan_deps_installed >> ..\scripts\install_status.txt
+)
+
+@>nul grep -c "conda_sd_esrgan_deps_installed" ..\scripts\install_status.txt
+@if "%ERRORLEVEL%" EQU "0" (
+    @echo "Packages necessary for ESRGAN (Resolution Upscaling) were already installed"
+) else (
+    @echo. & echo "Downloading packages necessary for ESRGAN (Resolution Upscaling).." & echo.
+
+    @set PYTHONNOUSERSITE=1
+
+    @call pip install -e git+https://github.com/xinntao/Real-ESRGAN#egg=realesrgan || (
+        @echo. & echo "Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    for /f "tokens=*" %%a in ('python -c "from basicsr.archs.rrdbnet_arch import RRDBNet; from realesrgan import RealESRGANer; print(42)"') do if "%%a" NEQ "42" (
+        @echo. & echo "Dependency test failed! Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+
+    @echo conda_sd_esrgan_deps_installed >> ..\scripts\install_status.txt
+)
 
 @>nul grep -c "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
 @if "%ERRORLEVEL%" EQU "0" (
@@ -61,43 +139,178 @@
 ) else (
     @echo. & echo "Downloading packages necessary for Stable Diffusion UI.." & echo.
 
-    @call conda install -c conda-forge -y --prefix env uvicorn fastapi && (
-        @echo conda_sd_ui_deps_installed >> ..\scripts\install_status.txt
-    ) || (
-        @set cmd_had_error=T
-    )
+    @set PYTHONNOUSERSITE=1
 
-    if "%ERRORLEVEL%" NEQ "0" (
-        @set cmd_had_error=T
-    )
-
-    if "%cmd_had_error%"=="T" (
-        echo "Error installing the packages necessary for Stable Diffusion UI. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues"
+    @call conda install -c conda-forge -y --prefix env uvicorn fastapi || (
+        echo "Error installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!"
         pause
         exit /b
     )
 )
 
+call WHERE uvicorn > .tmp
+@>nul grep -c "uvicorn" .tmp
+@if "%ERRORLEVEL%" NEQ "0" (
+    @echo. & echo "UI packages not found! Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+    pause
+    exit /b
+)
+
+@>nul grep -c "conda_sd_ui_deps_installed" ..\scripts\install_status.txt
+@if "%ERRORLEVEL%" NEQ "0" (
+    @echo conda_sd_ui_deps_installed >> ..\scripts\install_status.txt
+)
+
+
+
 @if exist "sd-v1-4.ckpt" (
-    echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
-) else (
+    for %%I in ("sd-v1-4.ckpt") do if "%%~zI" EQU "4265380512" (
+        echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
+    ) else (
+        for %%J in ("sd-v1-4.ckpt") do if "%%~zJ" EQU "7703807346" (
+            echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
+        ) else (
+            for %%K in ("sd-v1-4.ckpt") do if "%%~zK" EQU "7703810927" (
+                echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
+            ) else (
+                echo. & echo "The model file present at %cd%\sd-v1-4.ckpt is invalid. It is only %%~zK bytes in size. Re-downloading.." & echo.
+                del "sd-v1-4.ckpt"
+            )
+        )
+    )
+)
+
+@if not exist "sd-v1-4.ckpt" (
     @echo. & echo "Downloading data files (weights) for Stable Diffusion.." & echo.
 
-    @call curl -L https://me.cmdr2.org/stable-diffusion-ui/sd-v1-4.ckpt > sd-v1-4.ckpt
+    @call curl -L -k https://me.cmdr2.org/stable-diffusion-ui/sd-v1-4.ckpt > sd-v1-4.ckpt
 
-    @if not exist "sd-v1-4.ckpt" (
-        echo "Error downloading the data files (weights) for Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues"
+    @if exist "sd-v1-4.ckpt" (
+        for %%I in ("sd-v1-4.ckpt") do if "%%~zI" NEQ "4265380512" (
+            echo. & echo "Error: The downloaded model file was invalid! Bytes downloaded: %%~zI" & echo.
+            echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            pause
+            exit /b
+        )
+    ) else (
+        @echo. & echo "Error downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
         pause
         exit /b
     )
+)
 
+
+
+@if exist "GFPGANv1.3.pth" (
+    for %%I in ("GFPGANv1.3.pth") do if "%%~zI" EQU "348632874" (
+        echo "Data files (weights) necessary for GFPGAN (Face Correction) were already downloaded"
+    ) else (
+        echo. & echo "The GFPGAN model file present at %cd%\GFPGANv1.3.pth is invalid. It is only %%~zI bytes in size. Re-downloading.." & echo.
+        del "GFPGANv1.3.pth"
+    )
+)
+
+@if not exist "GFPGANv1.3.pth" (
+    @echo. & echo "Downloading data files (weights) for GFPGAN (Face Correction).." & echo.
+
+    @call curl -L -k https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth > GFPGANv1.3.pth
+
+    @if exist "GFPGANv1.3.pth" (
+        for %%I in ("GFPGANv1.3.pth") do if "%%~zI" NEQ "348632874" (
+            echo. & echo "Error: The downloaded GFPGAN model file was invalid! Bytes downloaded: %%~zI" & echo.
+            echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            pause
+            exit /b
+        )
+    ) else (
+        @echo. & echo "Error downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+)
+
+
+
+@if exist "RealESRGAN_x4plus.pth" (
+    for %%I in ("RealESRGAN_x4plus.pth") do if "%%~zI" EQU "67040989" (
+        echo "Data files (weights) necessary for ESRGAN (Resolution Upscaling) x4plus were already downloaded"
+    ) else (
+        echo. & echo "The GFPGAN model file present at %cd%\RealESRGAN_x4plus.pth is invalid. It is only %%~zI bytes in size. Re-downloading.." & echo.
+        del "RealESRGAN_x4plus.pth"
+    )
+)
+
+@if not exist "RealESRGAN_x4plus.pth" (
+    @echo. & echo "Downloading data files (weights) for ESRGAN (Resolution Upscaling) x4plus.." & echo.
+
+    @call curl -L -k https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth > RealESRGAN_x4plus.pth
+
+    @if exist "RealESRGAN_x4plus.pth" (
+        for %%I in ("RealESRGAN_x4plus.pth") do if "%%~zI" NEQ "67040989" (
+            echo. & echo "Error: The downloaded ESRGAN x4plus model file was invalid! Bytes downloaded: %%~zI" & echo.
+            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            pause
+            exit /b
+        )
+    ) else (
+        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+)
+
+
+
+@if exist "RealESRGAN_x4plus_anime_6B.pth" (
+    for %%I in ("RealESRGAN_x4plus_anime_6B.pth") do if "%%~zI" EQU "17938799" (
+        echo "Data files (weights) necessary for ESRGAN (Resolution Upscaling) x4plus_anime were already downloaded"
+    ) else (
+        echo. & echo "The GFPGAN model file present at %cd%\RealESRGAN_x4plus_anime_6B.pth is invalid. It is only %%~zI bytes in size. Re-downloading.." & echo.
+        del "RealESRGAN_x4plus_anime_6B.pth"
+    )
+)
+
+@if not exist "RealESRGAN_x4plus_anime_6B.pth" (
+    @echo. & echo "Downloading data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime.." & echo.
+
+    @call curl -L -k https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth > RealESRGAN_x4plus_anime_6B.pth
+
+    @if exist "RealESRGAN_x4plus_anime_6B.pth" (
+        for %%I in ("RealESRGAN_x4plus_anime_6B.pth") do if "%%~zI" NEQ "17938799" (
+            echo. & echo "Error: The downloaded ESRGAN x4plus_anime model file was invalid! Bytes downloaded: %%~zI" & echo.
+            echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+            pause
+            exit /b
+        )
+    ) else (
+        @echo. & echo "Error downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:" & echo "  1. Run this installer again." & echo "  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md" & echo "  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB" & echo "  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues" & echo "Thanks!" & echo.
+        pause
+        exit /b
+    )
+)
+
+
+
+@>nul grep -c "sd_install_complete" ..\scripts\install_status.txt
+@if "%ERRORLEVEL%" NEQ "0" (
     @echo sd_weights_downloaded >> ..\scripts\install_status.txt
     @echo sd_install_complete >> ..\scripts\install_status.txt
 )
 
 @echo. & echo "Stable Diffusion is ready!" & echo.
 
-@set SD_UI_PATH=%cd%\..\ui
+@set SD_DIR=%cd%
+
+@cd env\lib\site-packages
+@set PYTHONPATH=%SD_DIR%;%cd%
+@cd ..\..\..
+@echo PYTHONPATH=%PYTHONPATH%
+
+@cd ..
+@set SD_UI_PATH=%cd%\ui
+@cd stable-diffusion
+
+@call python --version
 
 @uvicorn server:app --app-dir "%SD_UI_PATH%" --port 9000 --host 0.0.0.0
 

scripts/on_sd_start.sh

@@ -1,12 +1,25 @@
+#!/bin/bash
+
+cp sd-ui-files/scripts/on_env_start.sh scripts/
+
 source installer/etc/profile.d/conda.sh
 
-if [ `grep -c sd_git_cloned scripts/install_status.txt` -gt "0" ]; then
+python -c "import os; import shutil; frm = 'sd-ui-files/ui/hotfix/9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'; dst = os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'transformers', '9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142'); shutil.copyfile(frm, dst) if os.path.exists(dst) else print(''); print('Hotfixed broken JSON file from OpenAI');"
+
+# Caution, this file will make your eyes and brain bleed. It's such an unholy mess.
+# Note to self: Please rewrite this in Python. For the sake of your own sanity.
+
+if [ -e "scripts/install_status.txt" ] && [ `grep -c sd_git_cloned scripts/install_status.txt` -gt "0" ]; then
     echo "Stable Diffusion's git repository was already installed. Updating.."
 
     cd stable-diffusion
 
     git reset --hard
     git pull
+    git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+
+    git apply ../ui/sd_internal/ddim_callback.patch
+    git apply ../ui/sd_internal/env_yaml.patch
 
     cd ..
 else
@@ -15,66 +28,282 @@ else
     if git clone https://github.com/basujindal/stable-diffusion.git ; then
         echo sd_git_cloned >> scripts/install_status.txt
     else
-        printf "\n\nError downloading Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\n\n"
+        printf "\n\nError downloading Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
+
+    cd stable-diffusion
+    git checkout f6cfebffa752ee11a7b07497b8529d5971de916c
+
+    git apply ../ui/sd_internal/ddim_callback.patch
+    git apply ../ui/sd_internal/env_yaml.patch
+
+    cd ..
 fi
 
 cd stable-diffusion
 
 if [ `grep -c conda_sd_env_created ../scripts/install_status.txt` -gt "0" ]; then
     echo "Packages necessary for Stable Diffusion were already installed"
+
+    conda activate ./env
 else
     printf "\n\nDownloading packages necessary for Stable Diffusion..\n"
     printf "\n\n***** This will take some time (depending on the speed of the Internet connection) and may appear to be stuck, but please be patient ***** ..\n\n"
 
+    # prevent conda from using packages from the user's home directory, to avoid conflicts
+    export PYTHONNOUSERSITE=1
+
     if conda env create --prefix env --force -f environment.yaml ; then
-        echo conda_sd_env_created >> ../scripts/install_status.txt
+        echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\n\n"
+        printf "\n\nError installing the packages necessary for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
+
+    conda activate ./env
+
+    if conda install -c conda-forge --prefix ./env -y antlr4-python3-runtime=4.8 ; then
+        echo "Installed. Testing.."
+    else
+        printf "\n\nError installing antlr4-python3-runtime for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    out_test=`python -c "import torch; import ldm; import transformers; import numpy; import antlr4; print(42)"`
+    if [ "$out_test" != "42" ]; then
+        printf "\n\nDependency test failed! Error installing the packages necessary for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    echo conda_sd_env_created >> ../scripts/install_status.txt
 fi
 
-conda activate ./env
+if [ `grep -c conda_sd_gfpgan_deps_installed ../scripts/install_status.txt` -gt "0" ]; then
+    echo "Packages necessary for GFPGAN (Face Correction) were already installed"
+else
+    printf "\n\nDownloading packages necessary for GFPGAN (Face Correction)..\n"
+
+    export PYTHONNOUSERSITE=1
+
+    if pip install -e git+https://github.com/TencentARC/GFPGAN#egg=GFPGAN ; then
+        echo "Installed. Testing.."
+    else
+        printf "\n\nError installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    out_test=`python -c "from gfpgan import GFPGANer; print(42)"`
+    if [ "$out_test" != "42" ]; then
+        printf "\n\nDependency test failed! Error installing the packages necessary for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    echo conda_sd_gfpgan_deps_installed >> ../scripts/install_status.txt
+fi
+
+if [ `grep -c conda_sd_esrgan_deps_installed ../scripts/install_status.txt` -gt "0" ]; then
+    echo "Packages necessary for ESRGAN (Resolution Upscaling) were already installed"
+else
+    printf "\n\nDownloading packages necessary for ESRGAN (Resolution Upscaling)..\n"
+
+    export PYTHONNOUSERSITE=1
+
+    if pip install -e git+https://github.com/xinntao/Real-ESRGAN#egg=realesrgan ; then
+        echo "Installed. Testing.."
+    else
+        printf "\n\nError installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    out_test=`python -c "from basicsr.archs.rrdbnet_arch import RRDBNet; from realesrgan import RealESRGANer; print(42)"`
+    if [ "$out_test" != "42" ]; then
+        printf "\n\nDependency test failed! Error installing the packages necessary for ESRGAN (Resolution Upscaling). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    echo conda_sd_esrgan_deps_installed >> ../scripts/install_status.txt
+fi
 
 if [ `grep -c conda_sd_ui_deps_installed ../scripts/install_status.txt` -gt "0" ]; then
     echo "Packages necessary for Stable Diffusion UI were already installed"
 else
     printf "\n\nDownloading packages necessary for Stable Diffusion UI..\n\n"
 
+    export PYTHONNOUSERSITE=1
+
     if conda install -c conda-forge --prefix ./env -y uvicorn fastapi ; then
-        echo conda_sd_ui_deps_installed >> ../scripts/install_status.txt
+        echo "Installed. Testing.."
     else
-        printf "\n\nError installing the packages necessary for Stable Diffusion UI. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\n\n"
+        printf "\n\nError installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    if ! command -v uvicorn &> /dev/null; then
+        printf "\n\nUI packages not found! Error installing the packages necessary for Stable Diffusion UI. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+
+    echo conda_sd_ui_deps_installed >> ../scripts/install_status.txt
+fi
+
+
+
+if [ -f "sd-v1-4.ckpt" ]; then
+    model_size=`ls -l sd-v1-4.ckpt | awk '{print $5}'`
+
+    if [ "$model_size" -eq "4265380512" ] || [ "$model_size" -eq "7703807346" ] || [ "$model_size" -eq "7703810927" ]; then
+        echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
+    else
+        printf "\n\nThe model file present at $PWD/sd-v1-4.ckpt is invalid. It is only $model_size bytes in size. Re-downloading.."
+        rm sd-v1-4.ckpt
+    fi
+fi
+
+if [ ! -f "sd-v1-4.ckpt" ]; then
+    echo "Downloading data files (weights) for Stable Diffusion.."
+
+    curl -L -k https://me.cmdr2.org/stable-diffusion-ui/sd-v1-4.ckpt > sd-v1-4.ckpt
+
+    if [ -f "sd-v1-4.ckpt" ]; then
+        model_size=`ls -l sd-v1-4.ckpt | awk '{print $5}'`
+        if [ ! "$model_size" == "4265380512" ]; then
+            printf "\n\nError: The downloaded model file was invalid! Bytes downloaded: $model_size\n\n"
+            printf "\n\nError downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+            read -p "Press any key to continue"
+            exit
+        fi
+    else
+        printf "\n\nError downloading the data files (weights) for Stable Diffusion. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
 fi
 
-if [ -f "sd-v1-4.ckpt" ]; then
-    echo "Data files (weights) necessary for Stable Diffusion were already downloaded"
-else
-    echo "Downloading data files (weights) for Stable Diffusion.."
 
-    curl -L https://me.cmdr2.org/stable-diffusion-ui/sd-v1-4.ckpt > sd-v1-4.ckpt
+if [ -f "GFPGANv1.3.pth" ]; then
+    model_size=`ls -l GFPGANv1.3.pth | awk '{print $5}'`
 
-    if [ ! -f "sd-v1-4.ckpt" ]; then
-        printf "\n\nError downloading the data files (weights) for Stable Diffusion. Please try re-running this installer. If it doesn't work, please copy the messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB or file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\n\n"
+    if [ "$model_size" -eq "348632874" ]; then
+        echo "Data files (weights) necessary for GFPGAN (Face Correction) were already downloaded"
+    else
+        printf "\n\nThe model file present at $PWD/GFPGANv1.3.pth is invalid. It is only $model_size bytes in size. Re-downloading.."
+        rm GFPGANv1.3.pth
+    fi
+fi
+
+if [ ! -f "GFPGANv1.3.pth" ]; then
+    echo "Downloading data files (weights) for GFPGAN (Face Correction).."
+
+    curl -L -k https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth > GFPGANv1.3.pth
+
+    if [ -f "GFPGANv1.3.pth" ]; then
+        model_size=`ls -l GFPGANv1.3.pth | awk '{print $5}'`
+        if [ ! "$model_size" -eq "348632874" ]; then
+            printf "\n\nError: The downloaded GFPGAN model file was invalid! Bytes downloaded: $model_size\n\n"
+            printf "\n\nError downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+            read -p "Press any key to continue"
+            exit
+        fi
+    else
+        printf "\n\nError downloading the data files (weights) for GFPGAN (Face Correction). Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
         read -p "Press any key to continue"
         exit
     fi
+fi
 
+
+if [ -f "RealESRGAN_x4plus.pth" ]; then
+    model_size=`ls -l RealESRGAN_x4plus.pth | awk '{print $5}'`
+
+    if [ "$model_size" -eq "67040989" ]; then
+        echo "Data files (weights) necessary for ESRGAN (Resolution Upscaling) x4plus were already downloaded"
+    else
+        printf "\n\nThe model file present at $PWD/RealESRGAN_x4plus.pth is invalid. It is only $model_size bytes in size. Re-downloading.."
+        rm RealESRGAN_x4plus.pth
+    fi
+fi
+
+if [ ! -f "RealESRGAN_x4plus.pth" ]; then
+    echo "Downloading data files (weights) for ESRGAN (Resolution Upscaling) x4plus.."
+
+    curl -L -k https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth > RealESRGAN_x4plus.pth
+
+    if [ -f "RealESRGAN_x4plus.pth" ]; then
+        model_size=`ls -l RealESRGAN_x4plus.pth | awk '{print $5}'`
+        if [ ! "$model_size" -eq "67040989" ]; then
+            printf "\n\nError: The downloaded ESRGAN x4plus model file was invalid! Bytes downloaded: $model_size\n\n"
+            printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+            read -p "Press any key to continue"
+            exit
+        fi
+    else
+        printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+fi
+
+
+if [ -f "RealESRGAN_x4plus_anime_6B.pth" ]; then
+    model_size=`ls -l RealESRGAN_x4plus_anime_6B.pth | awk '{print $5}'`
+
+    if [ "$model_size" -eq "17938799" ]; then
+        echo "Data files (weights) necessary for ESRGAN (Resolution Upscaling) x4plus_anime were already downloaded"
+    else
+        printf "\n\nThe model file present at $PWD/RealESRGAN_x4plus_anime_6B.pth is invalid. It is only $model_size bytes in size. Re-downloading.."
+        rm RealESRGAN_x4plus_anime_6B.pth
+    fi
+fi
+
+if [ ! -f "RealESRGAN_x4plus_anime_6B.pth" ]; then
+    echo "Downloading data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime.."
+
+    curl -L -k https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth > RealESRGAN_x4plus_anime_6B.pth
+
+    if [ -f "RealESRGAN_x4plus_anime_6B.pth" ]; then
+        model_size=`ls -l RealESRGAN_x4plus_anime_6B.pth | awk '{print $5}'`
+        if [ ! "$model_size" -eq "17938799" ]; then
+            printf "\n\nError: The downloaded ESRGAN x4plus_anime model file was invalid! Bytes downloaded: $model_size\n\n"
+            printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+            read -p "Press any key to continue"
+            exit
+        fi
+    else
+        printf "\n\nError downloading the data files (weights) for ESRGAN (Resolution Upscaling) x4plus_anime. Sorry about that, please try to:\n  1. Run this installer again.\n  2. If that doesn't fix it, please try the common troubleshooting steps at https://github.com/cmdr2/stable-diffusion-ui/blob/main/Troubleshooting.md\n  3. If those steps don't help, please copy *all* the error messages in this window, and ask the community at https://discord.com/invite/u9yhsFmEkB\n  4. If that doesn't solve the problem, please file an issue at https://github.com/cmdr2/stable-diffusion-ui/issues\nThanks!\n\n"
+        read -p "Press any key to continue"
+        exit
+    fi
+fi
+
+
+if [ `grep -c sd_install_complete ../scripts/install_status.txt` -gt "0" ]; then
     echo sd_weights_downloaded >> ../scripts/install_status.txt
     echo sd_install_complete >> ../scripts/install_status.txt
 fi
 
 printf "\n\nStable Diffusion is ready!\n\n"
 
-export SD_UI_PATH=`pwd`/../ui
+SD_PATH=`pwd`
+export PYTHONPATH="$SD_PATH;$SD_PATH/env/lib/python3.8/site-packages"
+echo "PYTHONPATH=$PYTHONPATH"
+
+cd ..
+export SD_UI_PATH=`pwd`/ui
+cd stable-diffusion
+
+python --version
 
 uvicorn server:app --app-dir "$SD_UI_PATH" --port 9000 --host 0.0.0.0
 
-read -p "Press any key to continue"
+read -p "Press any key to continue"

scripts/post_activate.sh

@@ -1,3 +1,5 @@
+#!/bin/bash
+
 conda-unpack
 
 source $CONDA_PREFIX/etc/profile.d/conda.sh

scripts/start.sh

@@ -1,3 +1,5 @@
+#!/bin/bash
+
 source installer/bin/activate
 
 conda-unpack

ui/hotfix/9c24e6cd9f499d02c4f21a033736dabd365962dc80fe3aeb57a8f85ea45a20a3.26fead7ea4f0f843f6eb4055dfd25693f1a71f3c6871b184042d4b126244e142

@@ -0,0 +1,171 @@
+{
+  "_name_or_path": "clip-vit-large-patch14/",
+  "architectures": [
+    "CLIPModel"
+  ],
+  "initializer_factor": 1.0,
+  "logit_scale_init_value": 2.6592,
+  "model_type": "clip",
+  "projection_dim": 768,
+  "text_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_dropout": 0.0,
+    "bad_words_ids": null,
+    "bos_token_id": 0,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout": 0.0,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": 2,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 77,
+    "min_length": 0,
+    "model_type": "clip_text_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 12,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 12,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": 1,
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim" : 768,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.16.0.dev0",
+    "use_bfloat16": false,
+    "vocab_size": 49408
+  },
+  "text_config_dict": {
+    "hidden_size": 768,
+    "intermediate_size": 3072,
+    "num_attention_heads": 12,
+    "num_hidden_layers": 12,
+    "projection_dim": 768
+  },
+  "torch_dtype": "float32",
+  "transformers_version": null,
+  "vision_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_dropout": 0.0,
+    "bad_words_ids": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout": 0.0,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 1024,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "image_size": 224,
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "min_length": 0,
+    "model_type": "clip_vision_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 16,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 24,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "patch_size": 14,
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim" : 768,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.16.0.dev0",
+    "use_bfloat16": false
+  },
+  "vision_config_dict": {
+    "hidden_size": 1024,
+    "intermediate_size": 4096,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 24,
+    "patch_size": 14,
+    "projection_dim": 768
+  }
+}

ui/modifiers.json

@@ -20,22 +20,42 @@
       "16-bit",
       "Anaglyph",
       "Anime",
-      "Cartoon",
+      "Art Nouveau",
+      "Bauhaus",
+      "Baroque",
       "CGI",
+      "Cartoon",
       "Comic Book",
       "Concept Art",
+      "Constructivist",
+      "Cubist",
       "Digital Art",
+      "Dadaist",
+      "Expressionist",
       "Fantasy",
+      "Fauvist",
+      "Figurative",
       "Graphic Novel",
+      "Geometric",
       "Hard Edge Painting",
       "Hydrodipped",
+      "Impressionistic",
       "Lithography",
+      "Manga",
+      "Minimalist",
       "Modern Art",
       "Mosaic",
       "Mural",
+      "Naive",
+      "Neoclassical",
       "Photo",
       "Realistic",
+      "Rococo",
+      "Romantic",
       "Street Art",
+      "Symbolist",
+      "Stuckist",
+      "Surrealist",
       "Visual Novel",
       "Watercolor"
     ]
@@ -67,7 +87,9 @@
     "Camera",
     [
       "Aerial View",
+      "Canon50",
       "Cinematic",
+      "Close-up",
       "Color Grading",
       "Dramatic",
       "Film Grain",
@@ -75,9 +97,13 @@
       "Glamor Shot",
       "Golden Hour",
       "HD",
+      "Landscape",
       "Lens Flare",
       "Macro",
       "Polaroid",
+      "Photoshoot",
+      "Portrait",
+      "Studio Lighting",
       "Vintage",
       "War Photography",
       "White Balance",
@@ -88,9 +114,13 @@
     "Color",
     [
       "Beautiful Lighting",
+      "Cold Color Palette",
       "Colorful",
+      "Dynamic Lighting",
       "Electric Colors",
       "Infrared",
+      "Pastel",
+      "Neon",
       "Synthwave",
       "Warm Color Palette"
     ]
@@ -99,42 +129,130 @@
     "Emotions",
     [
       "Angry",
+      "Bitter",
+      "Disgusted",
+      "Embarrassed",
       "Evil",
       "Excited",
-      "Good",
+      "Fear",
+      "Funny",
       "Happy",
+      "Horrifying",
       "Lonely",
-      "Sad"
+      "Sad",
+      "Serene",
+      "Surprised",
+      "Melancholic"
     ]
   ],
   [
     "Style of an artist or community",
     [
       "Artstation",
+      "trending on Artstation",
       "by Agnes Lawrence Pelton",
       "by Akihito Yoshida",
+      "by Alex Grey",
+      "by Alexander Jansson",
+      "by Alphonse Mucha",
       "by Andy Warhol",
+      "by Artgerm",
       "by Asaf Hanuka",
       "by Aubrey Beardsley",
+      "by Banksy",
+      "by Beeple",
       "by Ben Enwonwu",
+      "by Bob Eggleton",
       "by Caravaggio Michelangelo Merisi",
+      "by Caspar David Friedrich",
+      "by Chris Foss",
+      "by Claude Monet",
+      "by Dan Mumford",
       "by David Mann",
+      "by Diego Velázquez",
+      "by Disney Animation Studios",
+      "by Édouard Manet",
+      "by Esao Andrews",
       "by Frida Kahlo",
+      "by Gediminas Pranckevicius",
+      "by Georgia O'Keeffe",
+      "by Greg Rutkowski",
+      "by Gustave Doré",
+      "by Gustave Klimt",
       "by H.R. Giger",
-      "by Hayao Mizaki",
+      "by Hayao Miyazaki",
+      "by Henri Matisse",
+      "by HP Lovecraft",
       "by Ivan Shishkin",
+      "by Jack Kirby",
+      "by Jackson Pollock",
+      "by James Jean",
+      "by Jim Burns",
       "by Johannes Vermeer",
+      "by John William Waterhouse",
       "by Katsushika Hokusai",
+      "by Kim Tschang Yeul",
       "by Ko Young Hoon",
-      "by Leonardo Da Vinci",
+      "by Leonardo da Vinci",
       "by Lisa Frank",
-      "by Mahmoud Sai",
+      "by M.C. Escher",
+      "by Mahmoud Saïd",
+      "by Makoto Shinkai",
+      "by Marc Simonetti",
       "by Mark Brooks",
+      "by Michelangelo",
       "by Pablo Picasso",
+      "by Paul Klee",
+      "by Peter Mohrbacher",
+      "by Pierre-Auguste Renoir",
+      "by Pixar Animation Studios",
+      "by Rembrandt",
       "by Richard Dadd",
-      "by Salvador Dali",
+      "by Rossdraws",
+      "by Salvador Dalí",
+      "by Sam Does Arts",
+      "by Sandro Botticelli",
+      "by Ted Nasmith",
+      "by Ten Hundred",
+      "by Thomas Kinkade",
       "by Tivadar Csontváry Kosztka",
+      "by Victo Ngai",
+      "by Vincent Di Fate",
+      "by Vincent van Gogh",
+      "by Wes Anderson",
+      "by wlop",
       "by Yoshitaka Amano"
     ]
+  ],
+  [
+    "CGI Software",
+    [
+      "3D Model",
+      "3D Sculpt",
+      "3Ds Max Model",
+      "Blender Model",
+      "Cinema4d Model",
+      "Maya Model",
+      "Unreal Engine",
+      "Zbrush Sculpt"
+    ]
+  ],
+  [
+    "CGI Rendering",
+    [
+      "3D Render",
+      "Corona Render",
+      "Creature Design",
+      "Cycles Render",
+      "Detailed Render",
+      "Environment Design",
+      "Intricate Environment",
+      "LSD Render",
+      "Octane Render",
+      "PBR",
+      "Glass Caustics",
+      "Global Illumination",
+      "Subsurface Scattering"
+    ]
   ]
 ]

ui/sd_internal/__init__.py

@@ -1,6 +1,7 @@
 import json
 
 class Request:
+    session_id: str = "session"
     prompt: str = ""
     init_image: str = None # base64
     mask: str = None # base64
@@ -11,18 +12,43 @@ class Request:
     height: int = 512
     seed: int = 42
     prompt_strength: float = 0.8
+    sampler: str = None # "ddim", "plms", "heun", "euler", "euler_a", "dpm2", "dpm2_a", "lms"
     # allow_nsfw: bool = False
     precision: str = "autocast" # or "full"
     save_to_disk_path: str = None
     turbo: bool = True
     use_cpu: bool = False
     use_full_precision: bool = False
+    use_face_correction: str = None # or "GFPGANv1.3"
+    use_upscale: str = None # or "RealESRGAN_x4plus" or "RealESRGAN_x4plus_anime_6B"
+    show_only_filtered_image: bool = False
+
+    stream_progress_updates: bool = False
+    stream_image_progress: bool = False
+
+    def json(self):
+        return {
+            "session_id": self.session_id,
+            "prompt": self.prompt,
+            "num_outputs": self.num_outputs,
+            "num_inference_steps": self.num_inference_steps,
+            "guidance_scale": self.guidance_scale,
+            "width": self.width,
+            "height": self.height,
+            "seed": self.seed,
+            "prompt_strength": self.prompt_strength,
+            "sampler": self.sampler,
+            "use_face_correction": self.use_face_correction,
+            "use_upscale": self.use_upscale,
+        }
 
     def to_string(self):
         return f'''
+    session_id: {self.session_id}
     prompt: {self.prompt}
     seed: {self.seed}
     num_inference_steps: {self.num_inference_steps}
+    sampler: {self.sampler}
     guidance_scale: {self.guidance_scale}
     w: {self.width}
     h: {self.height}
@@ -30,12 +56,19 @@ class Request:
     save_to_disk_path: {self.save_to_disk_path}
     turbo: {self.turbo}
     use_cpu: {self.use_cpu}
-    use_full_precision: {self.use_full_precision}'''
+    use_full_precision: {self.use_full_precision}
+    use_face_correction: {self.use_face_correction}
+    use_upscale: {self.use_upscale}
+    show_only_filtered_image: {self.show_only_filtered_image}
+
+    stream_progress_updates: {self.stream_progress_updates}
+    stream_image_progress: {self.stream_image_progress}'''
 
 class Image:
     data: str # base64
     seed: int
     is_nsfw: bool
+    path_abs: str = None
 
     def __init__(self, data, seed):
         self.data = data
@@ -45,14 +78,17 @@ class Image:
         return {
             "data": self.data,
             "seed": self.seed,
+            "path_abs": self.path_abs,
         }
 
 class Response:
+    request: Request
     images: list
 
     def json(self):
         res = {
             "status": 'succeeded',
+            "request": self.request.json(),
             "output": [],
         }
 

ui/sd_internal/ddim_callback.patch

@@ -0,0 +1,332 @@
+diff --git a/optimizedSD/ddpm.py b/optimizedSD/ddpm.py
+index b967b55..35ef520 100644
+--- a/optimizedSD/ddpm.py
++++ b/optimizedSD/ddpm.py
+@@ -22,7 +22,7 @@ from ldm.util import exists, default, instantiate_from_config
+ from ldm.modules.diffusionmodules.util import make_beta_schedule
+ from ldm.modules.diffusionmodules.util import make_ddim_sampling_parameters, make_ddim_timesteps, noise_like
+ from ldm.modules.diffusionmodules.util import make_beta_schedule, extract_into_tensor, noise_like
+-from samplers import CompVisDenoiser, get_ancestral_step, to_d, append_dims,linear_multistep_coeff
++from .samplers import CompVisDenoiser, get_ancestral_step, to_d, append_dims,linear_multistep_coeff
+ 
+ def disabled_train(self):
+     """Overwrite model.train with this function to make sure train/eval mode
+@@ -506,6 +506,8 @@ class UNet(DDPM):
+ 
+         x_latent = noise if x0 is None else x0
+         # sampling
++        if sampler in ('ddim', 'dpm2', 'heun', 'dpm2_a', 'lms') and not hasattr(self, 'ddim_timesteps'):
++            self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=False)
+         
+         if sampler == "plms":
+             self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=False)
+@@ -528,39 +530,46 @@ class UNet(DDPM):
+         elif sampler == "ddim":
+             samples = self.ddim_sampling(x_latent, conditioning, S, unconditional_guidance_scale=unconditional_guidance_scale,
+                                          unconditional_conditioning=unconditional_conditioning,
+-                                         mask = mask,init_latent=x_T,use_original_steps=False)
++                                         mask = mask,init_latent=x_T,use_original_steps=False,
++                                         callback=callback, img_callback=img_callback)
+ 
+         elif sampler == "euler":
+             self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=False)
+             samples = self.euler_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
+         elif sampler == "euler_a":
+             self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=False)
+             samples = self.euler_ancestral_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
+ 
+         elif sampler == "dpm2":
+             samples = self.dpm_2_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
+         elif sampler == "heun":
+             samples = self.heun_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
+ 
+         elif sampler == "dpm2_a":
+             samples = self.dpm_2_ancestral_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
+ 
+ 
+         elif sampler == "lms":
+             samples = self.lms_sampling(self.alphas_cumprod,x_latent, S, conditioning, unconditional_conditioning=unconditional_conditioning,
+-                                        unconditional_guidance_scale=unconditional_guidance_scale)
++                                        unconditional_guidance_scale=unconditional_guidance_scale,
++                                        img_callback=img_callback)
++
++        yield from samples
+ 
+         if(self.turbo):
+             self.model1.to("cpu")
+             self.model2.to("cpu")
+ 
+-        return samples
+-
+     @torch.no_grad()
+     def plms_sampling(self, cond,b, img,
+                       ddim_use_original_steps=False,
+@@ -599,10 +608,10 @@ class UNet(DDPM):
+             old_eps.append(e_t)
+             if len(old_eps) >= 4:
+                 old_eps.pop(0)
+-            if callback: callback(i)
+-            if img_callback: img_callback(pred_x0, i)
++            if callback: yield from callback(i)
++            if img_callback: yield from img_callback(pred_x0, i)
+ 
+-        return img
++        yield from img_callback(img, len(iterator)-1)
+ 
+     @torch.no_grad()
+     def p_sample_plms(self, x, c, t, index, repeat_noise=False, use_original_steps=False, quantize_denoised=False,
+@@ -706,7 +715,8 @@ class UNet(DDPM):
+ 
+     @torch.no_grad()
+     def ddim_sampling(self, x_latent, cond, t_start, unconditional_guidance_scale=1.0, unconditional_conditioning=None,
+-               mask = None,init_latent=None,use_original_steps=False):
++               mask = None,init_latent=None,use_original_steps=False,
++               callback=None, img_callback=None):
+ 
+         timesteps = self.ddim_timesteps
+         timesteps = timesteps[:t_start]
+@@ -730,10 +740,13 @@ class UNet(DDPM):
+                                           unconditional_guidance_scale=unconditional_guidance_scale,
+                                           unconditional_conditioning=unconditional_conditioning)
+         
++            if callback: yield from callback(i)
++            if img_callback: yield from img_callback(x_dec, i)
++
+         if mask is not None:
+-            return x0 * mask + (1. - mask) * x_dec
++            x_dec = x0 * mask + (1. - mask) * x_dec
+ 
+-        return x_dec
++        yield from img_callback(x_dec, len(iterator)-1)
+ 
+ 
+     @torch.no_grad()
+@@ -779,13 +792,16 @@ class UNet(DDPM):
+ 
+ 
+     @torch.no_grad()
+-    def euler_sampling(self, ac, x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None,callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.):
++    def euler_sampling(self, ac, x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None,callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.,
++                        img_callback=None):
+         """Implements Algorithm 2 (Euler steps) from Karras et al. (2022)."""
+         extra_args = {} if extra_args is None else extra_args
+         cvd = CompVisDenoiser(ac)
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running Euler Sampling with {len(sigmas) - 1} timesteps")
++
+         s_in = x.new_ones([x.shape[0]]).half()
+         for i in trange(len(sigmas) - 1, disable=disable):
+             gamma = min(s_churn / (len(sigmas) - 1), 2 ** 0.5 - 1) if s_tmin <= sigmas[i] <= s_tmax else 0.
+@@ -807,13 +823,18 @@ class UNet(DDPM):
+             d = to_d(x, sigma_hat, denoised)
+             if callback is not None:
+                 callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigma_hat, 'denoised': denoised})
++
++            if img_callback: yield from img_callback(x, i)
++
+             dt = sigmas[i + 1] - sigma_hat
+             # Euler method
+             x = x + d * dt
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+ 
+     @torch.no_grad()
+-    def euler_ancestral_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None, callback=None, disable=None):
++    def euler_ancestral_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None, callback=None, disable=None,
++                        img_callback=None):
+         """Ancestral sampling with Euler method steps."""
+         extra_args = {} if extra_args is None else extra_args
+ 
+@@ -822,6 +843,8 @@ class UNet(DDPM):
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running Euler Ancestral Sampling with {len(sigmas) - 1} timesteps")
++
+         s_in = x.new_ones([x.shape[0]]).half()
+         for i in trange(len(sigmas) - 1, disable=disable):
+ 
+@@ -837,17 +860,22 @@ class UNet(DDPM):
+             sigma_down, sigma_up = get_ancestral_step(sigmas[i], sigmas[i + 1])
+             if callback is not None:
+                 callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigmas[i], 'denoised': denoised})
++
++            if img_callback: yield from img_callback(x, i)
++
+             d = to_d(x, sigmas[i], denoised)
+             # Euler method
+             dt = sigma_down - sigmas[i]
+             x = x + d * dt
+             x = x + torch.randn_like(x) * sigma_up
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+ 
+ 
+ 
+     @torch.no_grad()
+-    def heun_sampling(self, ac, x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.):
++    def heun_sampling(self, ac, x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.,
++                        img_callback=None):
+         """Implements Algorithm 2 (Heun steps) from Karras et al. (2022)."""
+         extra_args = {} if extra_args is None else extra_args
+ 
+@@ -855,6 +883,8 @@ class UNet(DDPM):
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running Heun Sampling with {len(sigmas) - 1} timesteps")
++
+ 
+         s_in = x.new_ones([x.shape[0]]).half()
+         for i in trange(len(sigmas) - 1, disable=disable):
+@@ -876,6 +906,9 @@ class UNet(DDPM):
+             d = to_d(x, sigma_hat, denoised)
+             if callback is not None:
+                 callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigma_hat, 'denoised': denoised})
++
++            if img_callback: yield from img_callback(x, i)
++
+             dt = sigmas[i + 1] - sigma_hat
+             if sigmas[i + 1] == 0:
+                 # Euler method
+@@ -895,11 +928,13 @@ class UNet(DDPM):
+                 d_2 = to_d(x_2, sigmas[i + 1], denoised_2)
+                 d_prime = (d + d_2) / 2
+                 x = x + d_prime * dt
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+ 
+ 
+     @torch.no_grad()
+-    def dpm_2_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None, callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.):
++    def dpm_2_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1,extra_args=None, callback=None, disable=None, s_churn=0., s_tmin=0., s_tmax=float('inf'), s_noise=1.,
++                        img_callback=None):
+         """A sampler inspired by DPM-Solver-2 and Algorithm 2 from Karras et al. (2022)."""
+         extra_args = {} if extra_args is None else extra_args
+ 
+@@ -907,6 +942,8 @@ class UNet(DDPM):
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running DPM2 Sampling with {len(sigmas) - 1} timesteps")
++
+         s_in = x.new_ones([x.shape[0]]).half()
+         for i in trange(len(sigmas) - 1, disable=disable):
+             gamma = min(s_churn / (len(sigmas) - 1), 2 ** 0.5 - 1) if s_tmin <= sigmas[i] <= s_tmax else 0.
+@@ -924,7 +961,7 @@ class UNet(DDPM):
+             e_t_uncond, e_t = (x_in  + eps * c_out).chunk(2)
+             denoised = e_t_uncond + unconditional_guidance_scale * (e_t - e_t_uncond)
+ 
+-
++            if img_callback: yield from img_callback(x, i)
+             
+             d = to_d(x, sigma_hat, denoised)
+             # Midpoint method, where the midpoint is chosen according to a rho=3 Karras schedule
+@@ -945,11 +982,13 @@ class UNet(DDPM):
+ 
+             d_2 = to_d(x_2, sigma_mid, denoised_2)
+             x = x + d_2 * dt_2
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+ 
+ 
+     @torch.no_grad()
+-    def dpm_2_ancestral_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None):
++    def dpm_2_ancestral_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None,
++                        img_callback=None):
+         """Ancestral sampling with DPM-Solver inspired second-order steps."""
+         extra_args = {} if extra_args is None else extra_args
+ 
+@@ -957,6 +996,8 @@ class UNet(DDPM):
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running DPM2 Ancestral Sampling with {len(sigmas) - 1} timesteps")
++
+         s_in = x.new_ones([x.shape[0]]).half()
+         for i in trange(len(sigmas) - 1, disable=disable):
+ 
+@@ -973,6 +1014,9 @@ class UNet(DDPM):
+             sigma_down, sigma_up = get_ancestral_step(sigmas[i], sigmas[i + 1])
+             if callback is not None:
+                 callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigmas[i], 'denoised': denoised})
++
++            if img_callback: yield from img_callback(x, i)
++
+             d = to_d(x, sigmas[i], denoised)
+             # Midpoint method, where the midpoint is chosen according to a rho=3 Karras schedule
+             sigma_mid = ((sigmas[i] ** (1 / 3) + sigma_down ** (1 / 3)) / 2) ** 3
+@@ -993,11 +1037,13 @@ class UNet(DDPM):
+             d_2 = to_d(x_2, sigma_mid, denoised_2)
+             x = x + d_2 * dt_2
+             x = x + torch.randn_like(x) * sigma_up
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+ 
+ 
+     @torch.no_grad()
+-    def lms_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None, order=4):
++    def lms_sampling(self,ac,x, S, cond, unconditional_conditioning = None, unconditional_guidance_scale = 1, extra_args=None, callback=None, disable=None, order=4,
++                        img_callback=None):
+         extra_args = {} if extra_args is None else extra_args
+         s_in = x.new_ones([x.shape[0]])
+ 
+@@ -1005,6 +1051,8 @@ class UNet(DDPM):
+         sigmas = cvd.get_sigmas(S)
+         x = x*sigmas[0]
+ 
++        print(f"Running LMS Sampling with {len(sigmas) - 1} timesteps")
++
+         ds = []
+         for i in trange(len(sigmas) - 1, disable=disable):
+ 
+@@ -1017,6 +1065,7 @@ class UNet(DDPM):
+             e_t_uncond, e_t = (x_in  + eps * c_out).chunk(2)
+             denoised = e_t_uncond + unconditional_guidance_scale * (e_t - e_t_uncond)
+ 
++            if img_callback: yield from img_callback(x, i)
+ 
+             d = to_d(x, sigmas[i], denoised)
+             ds.append(d)
+@@ -1027,4 +1076,5 @@ class UNet(DDPM):
+             cur_order = min(i + 1, order)
+             coeffs = [linear_multistep_coeff(cur_order, sigmas.cpu(), i, j) for j in range(cur_order)]
+             x = x + sum(coeff * d for coeff, d in zip(coeffs, reversed(ds)))
+-        return x
++
++        yield from img_callback(x, len(sigmas)-1)
+diff --git a/optimizedSD/openaimodelSplit.py b/optimizedSD/openaimodelSplit.py
+index abc3098..7a32ffe 100644
+--- a/optimizedSD/openaimodelSplit.py
++++ b/optimizedSD/openaimodelSplit.py
+@@ -13,7 +13,7 @@ from ldm.modules.diffusionmodules.util import (
+     normalization,
+     timestep_embedding,
+ )
+-from splitAttention import SpatialTransformer
++from .splitAttention import SpatialTransformer
+ 
+ 
+ class AttentionPool2d(nn.Module):

ui/sd_internal/env_yaml.patch

@@ -0,0 +1,13 @@
+diff --git a/environment.yaml b/environment.yaml
+index 7f25da8..306750f 100644
+--- a/environment.yaml
++++ b/environment.yaml
+@@ -23,6 +23,8 @@ dependencies:
+     - torch-fidelity==0.3.0
+     - transformers==4.19.2
+     - torchmetrics==0.6.0
++    - pywavelets==1.3.0
++    - pandas==1.4.4
+     - kornia==0.6
+     - -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers
+     - -e git+https://github.com/openai/CLIP.git@main#egg=clip

ui/sd_internal/runtime.py

@@ -1,8 +1,10 @@
+import json
 import os, re
+import traceback
 import torch
 import numpy as np
 from omegaconf import OmegaConf
-from PIL import Image
+from PIL import Image, ImageOps
 from tqdm import tqdm, trange
 from itertools import islice
 from einops import rearrange
@@ -15,25 +17,38 @@ from ldm.util import instantiate_from_config
 from optimizedSD.optimUtils import split_weighted_subprompts
 from transformers import logging
 
+from gfpgan import GFPGANer
+from basicsr.archs.rrdbnet_arch import RRDBNet
+from realesrgan import RealESRGANer
+
 import uuid
 
 logging.set_verbosity_error()
 
 # consts
 config_yaml = "optimizedSD/v1-inference.yaml"
+filename_regex = re.compile('[^a-zA-Z0-9]')
 
 # api stuff
 from . import Request, Response, Image as ResponseImage
 import base64
 from io import BytesIO
+#from colorama import Fore
 
 # local
-session_id = str(uuid.uuid4())
+stop_processing = False
+temp_images = {}
+
+ckpt_file = None
+gfpgan_file = None
+real_esrgan_file = None
 
-ckpt = None
 model = None
 modelCS = None
 modelFS = None
+model_gfpgan = None
+model_real_esrgan = None
+
 model_is_half = False
 model_fs_is_half = False
 device = None
@@ -42,16 +57,40 @@ precision = 'autocast'
 sampler_plms = None
 sampler_ddim = None
 
-# api
-def load_model(ckpt_to_use, device_to_use='cuda', turbo=False, unet_bs_to_use=1, precision_to_use='autocast', half_model_fs=False):
-    global ckpt, model, modelCS, modelFS, model_is_half, device, unet_bs, precision, model_fs_is_half
+has_valid_gpu = False
+force_full_precision = False
+try:
+    gpu = torch.cuda.current_device()
+    gpu_name = torch.cuda.get_device_name(gpu)
+    print('GPU detected: ', gpu_name)
 
-    ckpt = ckpt_to_use
-    device = device_to_use
-    precision = precision_to_use
+    force_full_precision = ('nvidia' in gpu_name.lower() or 'geforce' in gpu_name.lower()) and (' 1660' in gpu_name or ' 1650' in gpu_name) # otherwise these NVIDIA cards create green images
+    if force_full_precision:
+        print('forcing full precision on NVIDIA 16xx cards, to avoid green images. GPU detected: ', gpu_name)
+
+    mem_free, mem_total = torch.cuda.mem_get_info(gpu)
+    mem_total /= float(10**9)
+    if mem_total < 3.0:
+        print("GPUs with less than 3 GB of VRAM are not compatible with Stable Diffusion")
+        raise Exception()
+
+    has_valid_gpu = True
+except:
+    print('WARNING: No compatible GPU found. Using the CPU, but this will be very slow!')
+    pass
+
+def load_model_ckpt(ckpt_to_use, device_to_use='cuda', turbo=False, unet_bs_to_use=1, precision_to_use='autocast', half_model_fs=False):
+    global ckpt_file, model, modelCS, modelFS, model_is_half, device, unet_bs, precision, model_fs_is_half
+
+    ckpt_file = ckpt_to_use
+    device = device_to_use if has_valid_gpu else 'cpu'
+    precision = precision_to_use if not force_full_precision else 'full'
     unet_bs = unet_bs_to_use
 
-    sd = load_model_from_config(f"{ckpt}")
+    if device == 'cpu':
+        precision = 'full'
+
+    sd = load_model_from_config(f"{ckpt_file}.ckpt")
     li, lo = [], []
     for key, value in sd.items():
         sp = key.split(".")
@@ -101,29 +140,119 @@ def load_model(ckpt_to_use, device_to_use='cuda', turbo=False, unet_bs_to_use=1,
     else:
         model_fs_is_half = False
 
+    print('loaded ', ckpt_file, 'to', device, 'precision', precision)
+
+def load_model_gfpgan(gfpgan_to_use):
+    global gfpgan_file, model_gfpgan
+
+    if gfpgan_to_use is None:
+        return
+
+    gfpgan_file = gfpgan_to_use
+    model_path = gfpgan_to_use + ".pth"
+
+    if device == 'cpu':
+        model_gfpgan = GFPGANer(model_path=model_path, upscale=1, arch='clean', channel_multiplier=2, bg_upsampler=None, device=torch.device('cpu'))
+    else:
+        model_gfpgan = GFPGANer(model_path=model_path, upscale=1, arch='clean', channel_multiplier=2, bg_upsampler=None, device=torch.device('cuda'))
+
+    print('loaded ', gfpgan_to_use, 'to', device, 'precision', precision)
+
+def load_model_real_esrgan(real_esrgan_to_use):
+    global real_esrgan_file, model_real_esrgan
+
+    if real_esrgan_to_use is None:
+        return
+
+    real_esrgan_file = real_esrgan_to_use
+    model_path = real_esrgan_to_use + ".pth"
+
+    RealESRGAN_models = {
+        'RealESRGAN_x4plus': RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=4),
+        'RealESRGAN_x4plus_anime_6B': RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=6, num_grow_ch=32, scale=4)
+    }
+
+    model_to_use = RealESRGAN_models[real_esrgan_to_use]
+
+    if device == 'cpu':
+        model_real_esrgan = RealESRGANer(scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=False) # cpu does not support half
+        model_real_esrgan.device = torch.device('cpu')
+        model_real_esrgan.model.to('cpu')
+    else:
+        model_real_esrgan = RealESRGANer(scale=2, model_path=model_path, model=model_to_use, pre_pad=0, half=model_is_half)
+
+    model_real_esrgan.model.name = real_esrgan_to_use
+
+    print('loaded ', real_esrgan_to_use, 'to', device, 'precision', precision)
+
 def mk_img(req: Request):
-    global modelFS, device
+    try:
+        yield from do_mk_img(req)
+    except Exception as e:
+        print(traceback.format_exc())
+
+        gc()
+
+        if device != "cpu":
+            modelFS.to("cpu")
+            modelCS.to("cpu")
+
+            model.model1.to("cpu")
+            model.model2.to("cpu")
+
+        gc()
+
+        yield json.dumps({
+            "status": 'failed',
+            "detail": str(e)
+        })
+
+def do_mk_img(req: Request):
+    global model, modelCS, modelFS, device
+    global model_gfpgan, model_real_esrgan
+    global stop_processing
+
+    stop_processing = False
 
     res = Response()
+    res.request = req
     res.images = []
 
+    temp_images.clear()
+
     model.turbo = req.turbo
     if req.use_cpu:
-        device = 'cpu'
+        if device != 'cpu':
+            device = 'cpu'
 
-        if model_is_half:
-            print('reloading model for cpu')
-            load_model(ckpt, device)
+            if model_is_half:
+                del model, modelCS, modelFS
+                load_model_ckpt(ckpt_file, device)
+
+            load_model_gfpgan(gfpgan_file)
+            load_model_real_esrgan(real_esrgan_file)
     else:
-        device = 'cuda'
+        if has_valid_gpu:
+            prev_device = device
+            device = 'cuda'
 
-        if (precision == 'autocast' and (req.use_full_precision or not model_is_half)) or \
-            (precision == 'full' and not req.use_full_precision) or \
-            (req.init_image is None and model_fs_is_half) or \
-            (req.init_image is not None and not model_fs_is_half):
+            if (precision == 'autocast' and (req.use_full_precision or not model_is_half)) or \
+                (precision == 'full' and not req.use_full_precision and not force_full_precision) or \
+                (req.init_image is None and model_fs_is_half) or \
+                (req.init_image is not None and not model_fs_is_half and not force_full_precision):
 
-            print('reloading model for cuda')
-            load_model(ckpt, device, model.turbo, unet_bs, ('full' if req.use_full_precision else 'autocast'), half_model_fs=(req.init_image is not None and not req.use_full_precision))
+                del model, modelCS, modelFS
+                load_model_ckpt(ckpt_file, device, req.turbo, unet_bs, ('full' if req.use_full_precision else 'autocast'), half_model_fs=(req.init_image is not None and not req.use_full_precision))
+
+                if prev_device != device:
+                    load_model_gfpgan(gfpgan_file)
+                    load_model_real_esrgan(real_esrgan_file)
+
+    if req.use_face_correction != gfpgan_file:
+        load_model_gfpgan(req.use_face_correction)
+
+    if req.use_upscale != real_esrgan_file:
+        load_model_real_esrgan(req.use_upscale)
 
     model.cdevice = device
     modelCS.cond_stage_model.device = device
@@ -142,10 +271,16 @@ def mk_img(req: Request):
     opt_strength = req.prompt_strength
     opt_save_to_disk_path = req.save_to_disk_path
     opt_init_img = req.init_image
+    opt_use_face_correction = req.use_face_correction
+    opt_use_upscale = req.use_upscale
+    opt_show_only_filtered = req.show_only_filtered_image
     opt_format = 'png'
+    opt_sampler_name = req.sampler
 
     print(req.to_string(), '\n    device', device)
 
+    print('\n\n    Using precision:', precision)
+
     seed_everything(opt_seed)
 
     batch_size = opt_n_samples
@@ -158,6 +293,8 @@ def mk_img(req: Request):
     else:
         precision_scope = nullcontext
 
+    mask = None
+
     if req.init_image is None:
         handler = _txt2img
 
@@ -166,7 +303,7 @@ def mk_img(req: Request):
     else:
         handler = _img2img
 
-        init_image = load_img(req.init_image)
+        init_image = load_img(req.init_image, opt_W, opt_H)
         init_image = init_image.to(device)
 
         if device != "cpu" and precision == "autocast":
@@ -177,18 +314,22 @@ def mk_img(req: Request):
         init_image = repeat(init_image, '1 ... -> b ...', b=batch_size)
         init_latent = modelFS.get_first_stage_encoding(modelFS.encode_first_stage(init_image))  # move to latent space
 
-        if device != "cpu":
-            mem = torch.cuda.memory_allocated() / 1e6
-            modelFS.to("cpu")
-            while torch.cuda.memory_allocated() / 1e6 >= mem:
-                time.sleep(1)
+        if req.mask is not None:
+            mask = load_mask(req.mask, opt_W, opt_H, init_latent.shape[2], init_latent.shape[3], True).to(device)
+            mask = mask[0][0].unsqueeze(0).repeat(4, 1, 1).unsqueeze(0)
+            mask = repeat(mask, '1 ... -> b ...', b=batch_size)
+
+            if device != "cpu" and precision == "autocast":
+                mask = mask.half()
+
+        move_fs_to_cpu()
 
         assert 0. <= opt_strength <= 1., 'can only work with strength in [0.0, 1.0]'
         t_enc = int(opt_strength * opt_ddim_steps)
         print(f"target t_enc is {t_enc} steps")
 
     if opt_save_to_disk_path is not None:
-        session_out_path = os.path.join(opt_save_to_disk_path, 'session-' + session_id)
+        session_out_path = os.path.join(opt_save_to_disk_path, req.session_id)
         os.makedirs(session_out_path, exist_ok=True)
     else:
         session_out_path = None
@@ -198,9 +339,6 @@ def mk_img(req: Request):
         for n in trange(opt_n_iter, desc="Sampling"):
             for prompts in tqdm(data, desc="data"):
 
-                if opt_save_to_disk_path is not None:
-                    base_count = len(os.listdir(session_out_path))
-
                 with precision_scope("cuda"):
                     modelCS.to(device)
                     uc = None
@@ -222,54 +360,161 @@ def mk_img(req: Request):
                     else:
                         c = modelCS.get_learned_conditioning(prompts)
 
-                    # run the handler
-                    if handler == _txt2img:
-                        x_samples = _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, None, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed)
-                    else:
-                        x_samples = _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed)
-
                     modelFS.to(device)
 
+                    partial_x_samples = None
+                    def img_callback(x_samples, i):
+                        nonlocal partial_x_samples
+
+                        partial_x_samples = x_samples
+
+                        if req.stream_progress_updates:
+                            n_steps = opt_ddim_steps if req.init_image is None else t_enc
+                            progress = {"step": i, "total_steps": n_steps}
+
+                            if req.stream_image_progress and i % 5 == 0:
+                                partial_images = []
+
+                                for i in range(batch_size):
+                                    x_samples_ddim = modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
+                                    x_sample = torch.clamp((x_samples_ddim + 1.0) / 2.0, min=0.0, max=1.0)
+                                    x_sample = 255.0 * rearrange(x_sample[0].cpu().numpy(), "c h w -> h w c")
+                                    x_sample = x_sample.astype(np.uint8)
+                                    img = Image.fromarray(x_sample)
+                                    buf = BytesIO()
+                                    img.save(buf, format='JPEG')
+                                    buf.seek(0)
+
+                                    del img, x_sample, x_samples_ddim
+                                    # don't delete x_samples, it is used in the code that called this callback
+
+                                    temp_images[str(req.session_id) + '/' + str(i)] = buf
+                                    partial_images.append({'path': f'/image/tmp/{req.session_id}/{i}'})
+
+                                progress['output'] = partial_images
+
+                            yield json.dumps(progress)
+
+                        if stop_processing:
+                            raise UserInitiatedStop("User requested that we stop processing")
+
+                    # run the handler
+                    try:
+                        if handler == _txt2img:
+                            x_samples = _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, None, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed, img_callback, mask, opt_sampler_name)
+                        else:
+                            x_samples = _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed, img_callback, mask)
+
+                        yield from x_samples
+
+                        x_samples = partial_x_samples
+                    except UserInitiatedStop:
+                        if partial_x_samples is None:
+                            continue
+
+                        x_samples = partial_x_samples
+
                     print("saving images")
                     for i in range(batch_size):
 
                         x_samples_ddim = modelFS.decode_first_stage(x_samples[i].unsqueeze(0))
                         x_sample = torch.clamp((x_samples_ddim + 1.0) / 2.0, min=0.0, max=1.0)
                         x_sample = 255.0 * rearrange(x_sample[0].cpu().numpy(), "c h w -> h w c")
-                        img = Image.fromarray(x_sample.astype(np.uint8))
+                        x_sample = x_sample.astype(np.uint8)
+                        img = Image.fromarray(x_sample)
 
-                        img_data = img_to_base64_str(img)
-                        res.images.append(ResponseImage(data=img_data, seed=opt_seed))
+                        has_filters =   (opt_use_face_correction is not None and opt_use_face_correction.startswith('GFPGAN')) or \
+                                        (opt_use_upscale is not None and opt_use_upscale.startswith('RealESRGAN'))
+
+                        return_orig_img = not has_filters or not opt_show_only_filtered
+
+                        if stop_processing:
+                            return_orig_img = True
 
                         if opt_save_to_disk_path is not None:
-                            prompt_flattened = "_".join(re.split(":| ", prompts[0]))
-                            prompt_flattened = prompt_flattened[:150]
+                            prompt_flattened = filename_regex.sub('_', prompts[0])
+                            prompt_flattened = prompt_flattened[:50]
 
-                            file_path = f"sd_{prompt_flattened}_Seed-{opt_seed}_Steps-{opt_ddim_steps}_Guidance-{opt_scale}_{base_count:05}"
+                            img_id = str(uuid.uuid4())[-8:]
+
+                            file_path = f"{prompt_flattened}_{img_id}"
                             img_out_path = os.path.join(session_out_path, f"{file_path}.{opt_format}")
                             meta_out_path = os.path.join(session_out_path, f"{file_path}.txt")
 
-                            metadata = f"{prompts[0]}\nSeed: {opt_seed}\nSteps: {opt_ddim_steps}\nGuidance Scale: {opt_scale}"
-                            img.save(img_out_path)
-                            with open(meta_out_path, 'w') as f:
-                                f.write(metadata)
+                            if return_orig_img:
+                                save_image(img, img_out_path)
 
-                            base_count += 1
+                            save_metadata(meta_out_path, prompts, opt_seed, opt_W, opt_H, opt_ddim_steps, opt_scale, opt_strength, opt_use_face_correction, opt_use_upscale, opt_sampler_name)
+
+                        if return_orig_img:
+                            img_data = img_to_base64_str(img)
+                            res_image_orig = ResponseImage(data=img_data, seed=opt_seed)
+                            res.images.append(res_image_orig)
+
+                            if opt_save_to_disk_path is not None:
+                                res_image_orig.path_abs = img_out_path
+
+                        del img
+
+                        if has_filters and not stop_processing:
+                            print('Applying filters..')
+
+                            gc()
+                            filters_applied = []
+
+                            if opt_use_face_correction:
+                                _, _, output = model_gfpgan.enhance(x_sample[:,:,::-1], has_aligned=False, only_center_face=False, paste_back=True)
+                                x_sample = output[:,:,::-1]
+                                filters_applied.append(opt_use_face_correction)
+
+                            if opt_use_upscale:
+                                output, _ = model_real_esrgan.enhance(x_sample[:,:,::-1])
+                                x_sample = output[:,:,::-1]
+                                filters_applied.append(opt_use_upscale)
+
+                            filtered_image = Image.fromarray(x_sample)
+
+                            filtered_img_data = img_to_base64_str(filtered_image)
+                            res_image_filtered = ResponseImage(data=filtered_img_data, seed=opt_seed)
+                            res.images.append(res_image_filtered)
+
+                            filters_applied = "_".join(filters_applied)
+
+                            if opt_save_to_disk_path is not None:
+                                filtered_img_out_path = os.path.join(session_out_path, f"{file_path}_{filters_applied}.{opt_format}")
+                                save_image(filtered_image, filtered_img_out_path)
+                                res_image_filtered.path_abs = filtered_img_out_path
+
+                            del filtered_image
 
                         seeds += str(opt_seed) + ","
                         opt_seed += 1
 
-                    if device != "cpu":
-                        mem = torch.cuda.memory_allocated() / 1e6
-                        modelFS.to("cpu")
-                        while torch.cuda.memory_allocated() / 1e6 >= mem:
-                            time.sleep(1)
-                    del x_samples
+                    move_fs_to_cpu()
+                    gc()
+                    del x_samples, x_samples_ddim, x_sample
                     print("memory_final = ", torch.cuda.memory_allocated() / 1e6)
 
-    return res
+    print('Task completed')
 
-def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed):
+    yield json.dumps(res.json())
+
+def save_image(img, img_out_path):
+    try:
+        img.save(img_out_path)
+    except:
+        print('could not save the file', traceback.format_exc())
+
+def save_metadata(meta_out_path, prompts, opt_seed, opt_W, opt_H, opt_ddim_steps, opt_scale, opt_prompt_strength, opt_correct_face, opt_upscale, sampler_name):
+    metadata = f"{prompts[0]}\nWidth: {opt_W}\nHeight: {opt_H}\nSeed: {opt_seed}\nSteps: {opt_ddim_steps}\nGuidance Scale: {opt_scale}\nPrompt Strength: {opt_prompt_strength}\nUse Face Correction: {opt_correct_face}\nUse Upscaling: {opt_upscale}\nSampler: {sampler_name}"
+
+    try:
+        with open(meta_out_path, 'w') as f:
+            f.write(metadata)
+    except:
+        print('could not save the file', traceback.format_exc())
+
+def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code, opt_C, opt_f, opt_ddim_eta, c, uc, opt_seed, img_callback, mask, sampler_name):
     shape = [opt_n_samples, opt_C, opt_H // opt_f, opt_W // opt_f]
 
     if device != "cpu":
@@ -278,6 +523,9 @@ def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code,
         while torch.cuda.memory_allocated() / 1e6 >= mem:
             time.sleep(1)
 
+    if sampler_name == 'ddim':
+        model.make_schedule(ddim_num_steps=opt_ddim_steps, ddim_eta=opt_ddim_eta, verbose=False)
+
     samples_ddim = model.sample(
         S=opt_ddim_steps,
         conditioning=c,
@@ -288,12 +536,14 @@ def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code,
         unconditional_conditioning=uc,
         eta=opt_ddim_eta,
         x_T=start_code,
-        sampler = 'plms',
+        img_callback=img_callback,
+        mask=mask,
+        sampler = sampler_name,
     )
 
-    return samples_ddim
+    yield from samples_ddim
 
-def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed):
+def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, opt_ddim_eta, opt_seed, img_callback, mask):
     # encode (scaled latent)
     z_enc = model.stochastic_encode(
         init_latent,
@@ -302,6 +552,8 @@ def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, o
         opt_ddim_eta,
         opt_ddim_steps,
     )
+    x_T = None if mask is None else init_latent
+
     # decode it
     samples_ddim = model.sample(
         t_enc,
@@ -309,10 +561,27 @@ def _img2img(init_latent, t_enc, batch_size, opt_scale, c, uc, opt_ddim_steps, o
         z_enc,
         unconditional_guidance_scale=opt_scale,
         unconditional_conditioning=uc,
+        img_callback=img_callback,
+        mask=mask,
+        x_T=x_T,
         sampler = 'ddim'
     )
 
-    return samples_ddim
+    yield from samples_ddim
+
+def move_fs_to_cpu():
+    if device != "cpu":
+        mem = torch.cuda.memory_allocated() / 1e6
+        modelFS.to("cpu")
+        while torch.cuda.memory_allocated() / 1e6 >= mem:
+            time.sleep(1)
+
+def gc():
+    if device == 'cpu':
+        return
+
+    torch.cuda.empty_cache()
+    torch.cuda.ipc_collect()
 
 # internal
 
@@ -330,18 +599,48 @@ def load_model_from_config(ckpt, verbose=False):
     return sd
 
 # utils
+class UserInitiatedStop(Exception):
+    pass
 
-def load_img(img_str):
+def load_img(img_str, w0, h0):
     image = base64_str_to_img(img_str).convert("RGB")
     w, h = image.size
     print(f"loaded input image of size ({w}, {h}) from base64")
+    if h0 is not None and w0 is not None:
+        h, w = h0, w0
+
     w, h = map(lambda x: x - x % 64, (w, h))  # resize to integer multiple of 64
-    image = image.resize((w, h), resample=Image.LANCZOS)
+    image = image.resize((w, h), resample=Image.Resampling.LANCZOS)
     image = np.array(image).astype(np.float32) / 255.0
     image = image[None].transpose(0, 3, 1, 2)
     image = torch.from_numpy(image)
     return 2.*image - 1.
 
+def load_mask(mask_str, h0, w0, newH, newW, invert=False):
+    image = base64_str_to_img(mask_str).convert("RGB")
+    w, h = image.size
+    print(f"loaded input mask of size ({w}, {h})")
+
+    if invert:
+        print("inverted")
+        image = ImageOps.invert(image)
+        # where_0, where_1 = np.where(image == 0), np.where(image == 255)
+        # image[where_0], image[where_1] = 255, 0
+
+    if h0 is not None and w0 is not None:
+        h, w = h0, w0
+
+    w, h = map(lambda x: x - x % 64, (w, h))  # resize to integer multiple of 64
+
+    print(f"New mask size ({w}, {h})")
+    image = image.resize((newW, newH), resample=Image.Resampling.LANCZOS)
+    image = np.array(image)
+
+    image = image.astype(np.float32) / 255.0
+    image = image[None].transpose(0, 3, 1, 2)
+    image = torch.from_numpy(image)
+    return image
+
 # https://stackoverflow.com/a/61114178
 def img_to_base64_str(img):
     buffered = BytesIO()

ui/server.py

@@ -1,3 +1,4 @@
+import json
 import traceback
 
 import sys
@@ -9,11 +10,15 @@ print('started in ', SCRIPT_DIR)
 SD_UI_DIR = os.getenv('SD_UI_PATH', None)
 sys.path.append(os.path.dirname(SD_UI_DIR))
 
+CONFIG_DIR = os.path.join(SD_UI_DIR, '..', 'scripts')
+
 OUTPUT_DIRNAME = "Stable Diffusion UI" # in the user's home folder
 
 from fastapi import FastAPI, HTTPException
-from starlette.responses import FileResponse
+from fastapi.staticfiles import StaticFiles
+from starlette.responses import FileResponse, StreamingResponse
 from pydantic import BaseModel
+import logging
 
 from sd_internal import Request, Response
 
@@ -27,6 +32,7 @@ outpath = os.path.join(os.path.expanduser("~"), OUTPUT_DIRNAME)
 
 # defaults from https://huggingface.co/blog/stable_diffusion
 class ImageRequest(BaseModel):
+    session_id: str = "session"
     prompt: str = ""
     init_image: str = None # base64
     mask: str = None # base64
@@ -37,15 +43,28 @@ class ImageRequest(BaseModel):
     height: int = 512
     seed: int = 42
     prompt_strength: float = 0.8
+    sampler: str = None # "ddim", "plms", "heun", "euler", "euler_a", "dpm2", "dpm2_a", "lms"
     # allow_nsfw: bool = False
-    save_to_disk: bool = False
+    save_to_disk_path: str = None
     turbo: bool = True
     use_cpu: bool = False
     use_full_precision: bool = False
+    use_face_correction: str = None # or "GFPGANv1.3"
+    use_upscale: str = None # or "RealESRGAN_x4plus" or "RealESRGAN_x4plus_anime_6B"
+    show_only_filtered_image: bool = False
+
+    stream_progress_updates: bool = False
+    stream_image_progress: bool = False
+
+class SetAppConfigRequest(BaseModel):
+    update_branch: str = "main"
+
+app.mount('/media', StaticFiles(directory=os.path.join(SD_UI_DIR, 'media/')), name="media")
 
 @app.get('/')
 def read_root():
-    return FileResponse(os.path.join(SD_UI_DIR, 'index.html'))
+    headers = {"Cache-Control": "no-cache, no-store, must-revalidate", "Pragma": "no-cache", "Expires": "0"}
+    return FileResponse(os.path.join(SD_UI_DIR, 'index.html'), headers=headers)
 
 @app.get('/ping')
 async def ping():
@@ -61,7 +80,7 @@ async def ping():
         model_is_loading = True
 
         from sd_internal import runtime
-        runtime.load_model(ckpt_to_use="sd-v1-4.ckpt")
+        runtime.load_model_ckpt(ckpt_to_use="sd-v1-4")
 
         model_loaded = True
         model_is_loading = False
@@ -72,10 +91,11 @@ async def ping():
         return HTTPException(status_code=500, detail=str(e))
 
 @app.post('/image')
-async def image(req : ImageRequest):
+def image(req : ImageRequest):
     from sd_internal import runtime
 
     r = Request()
+    r.session_id = req.session_id
     r.prompt = req.prompt
     r.init_image = req.init_image
     r.mask = req.mask
@@ -86,25 +106,103 @@ async def image(req : ImageRequest):
     r.height = req.height
     r.seed = req.seed
     r.prompt_strength = req.prompt_strength
+    r.sampler = req.sampler
     # r.allow_nsfw = req.allow_nsfw
     r.turbo = req.turbo
     r.use_cpu = req.use_cpu
     r.use_full_precision = req.use_full_precision
+    r.save_to_disk_path = req.save_to_disk_path
+    r.use_upscale: str = req.use_upscale
+    r.use_face_correction = req.use_face_correction
+    r.show_only_filtered_image = req.show_only_filtered_image
 
-    if req.save_to_disk:
-        r.save_to_disk_path = outpath
+    r.stream_progress_updates = True # the underlying implementation only supports streaming
+    r.stream_image_progress = req.stream_image_progress
 
     try:
-        res: Response = runtime.mk_img(r)
+        if not req.stream_progress_updates:
+            r.stream_image_progress = False
 
-        return res.json()
+        res = runtime.mk_img(r)
+
+        if req.stream_progress_updates:
+            return StreamingResponse(res, media_type='application/json')
+        else: # compatibility mode: buffer the streaming responses, and return the last one
+            last_result = None
+
+            for result in res:
+                last_result = result
+
+            return json.loads(last_result)
     except Exception as e:
         print(traceback.format_exc())
         return HTTPException(status_code=500, detail=str(e))
 
-@app.get('/media/ding.mp3')
-def read_ding():
-    return FileResponse(os.path.join(SD_UI_DIR, 'media/ding.mp3'))
+@app.get('/image/stop')
+def stop():
+    try:
+        if model_is_loading:
+            return {'ERROR'}
+
+        from sd_internal import runtime
+        runtime.stop_processing = True
+
+        return {'OK'}
+    except Exception as e:
+        print(traceback.format_exc())
+        return HTTPException(status_code=500, detail=str(e))
+
+@app.get('/image/tmp/{session_id}/{img_id}')
+def get_image(session_id, img_id):
+    from sd_internal import runtime
+    buf = runtime.temp_images[session_id + '/' + img_id]
+    buf.seek(0)
+    return StreamingResponse(buf, media_type='image/jpeg')
+
+@app.post('/app_config')
+async def setAppConfig(req : SetAppConfigRequest):
+    try:
+        config = {
+            'update_branch': req.update_branch
+        }
+
+        config_json_str = json.dumps(config)
+        config_bat_str = f'@set update_branch={req.update_branch}'
+        config_sh_str = f'export update_branch={req.update_branch}'
+
+        config_json_path = os.path.join(CONFIG_DIR, 'config.json')
+        config_bat_path = os.path.join(CONFIG_DIR, 'config.bat')
+        config_sh_path = os.path.join(CONFIG_DIR, 'config.sh')
+
+        with open(config_json_path, 'w') as f:
+            f.write(config_json_str)
+
+        with open(config_bat_path, 'w') as f:
+            f.write(config_bat_str)
+
+        with open(config_sh_path, 'w') as f:
+            f.write(config_sh_str)
+
+        return {'OK'}
+    except Exception as e:
+        print(traceback.format_exc())
+        return HTTPException(status_code=500, detail=str(e))
+
+@app.get('/app_config')
+def getAppConfig():
+    try:
+        config_json_path = os.path.join(CONFIG_DIR, 'config.json')
+
+        if not os.path.exists(config_json_path):
+            return HTTPException(status_code=500, detail="No config file")
+
+        with open(config_json_path, 'r') as f:
+            config_json_str = f.read()
+            config = json.loads(config_json_str)
+            return config
+    except Exception as e:
+        print(traceback.format_exc())
+        return HTTPException(status_code=500, detail=str(e))
 
 @app.get('/modifiers.json')
 def read_modifiers():
@@ -114,5 +212,12 @@ def read_modifiers():
 def read_home_dir():
     return {outpath}
 
+# don't log /ping requests
+class HealthCheckLogFilter(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        return record.getMessage().find('/ping') == -1
+
+logging.getLogger('uvicorn.access').addFilter(HealthCheckLogFilter())
+
 # start the browser ui
 import webbrowser; webbrowser.open('http://localhost:9000')