Merge branch 'beta' into pause

CHANGES.md

@@ -2,12 +2,12 @@
 
 ## v2.4
 ### Major Changes
-- **Automatic scanning for malicious model files** - using `picklescan`, and support for `safetensor` model format. Thanks @JeLuf
-- **Support for custom VAE models**. You can place your VAE files in the `models/vae` folder, and refresh the browser page to use them. More info: https://github.com/cmdr2/stable-diffusion-ui/wiki/VAE-Variational-Auto-Encoder
-- **Use pre-trained hypernetworks** - for improving the quality of images. Thanks @C0bra5
-- **Experimental support for multiple GPUs!** It should work automatically. Just open one browser tab per GPU, and spread your tasks across your GPUs. For e.g. open our UI in two browser tabs if you have two GPUs. You can customize which GPUs it should use in the "Settings" tab, otherwise let it automatically pick the best GPUs. Thanks @madrang . More info: https://github.com/cmdr2/stable-diffusion-ui/wiki/Run-on-Multiple-GPUs
-- **Image Editor** - for drawing simple images for guiding the AI. Thanks @mdiller
 - **Allow reordering the task queue** (by dragging and dropping tasks). Thanks @madrang
+- **Automatic scanning for malicious model files** - using `picklescan`, and support for `safetensor` model format. Thanks @JeLuf
+- **Image Editor** - for drawing simple images for guiding the AI. Thanks @mdiller
+- **Use pre-trained hypernetworks** - for improving the quality of images. Thanks @C0bra5
+- **Support for custom VAE models**. You can place your VAE files in the `models/vae` folder, and refresh the browser page to use them. More info: https://github.com/cmdr2/stable-diffusion-ui/wiki/VAE-Variational-Auto-Encoder
+- **Experimental support for multiple GPUs!** It should work automatically. Just open one browser tab per GPU, and spread your tasks across your GPUs. For e.g. open our UI in two browser tabs if you have two GPUs. You can customize which GPUs it should use in the "Settings" tab, otherwise let it automatically pick the best GPUs. Thanks @madrang . More info: https://github.com/cmdr2/stable-diffusion-ui/wiki/Run-on-Multiple-GPUs
 - **Cleaner UI design** - Show settings and help in new tabs, instead of dropdown popups (which were buggy). Thanks @mdiller
 - **Progress bar.** Thanks @mdiller
 - **Custom Image Modifiers** - You can now save your custom image modifiers! Your saved modifiers can include special characters like `{}, (), [], |`
@@ -27,6 +27,7 @@
 - Support loading models in the safetensor format, for improved safety
 
 ### Detailed changelog
+* 2.4.19 - 17 Dec 2022 - Add Undo/Redo buttons in the Image Editor. Thanks @JeLuf
 * 2.4.19 - 10 Dec 2022 - Show init img in task list
 * 2.4.19 - 7 Dec 2022 - Use pre-trained hypernetworks while generating images. Thanks @C0bra5
 * 2.4.19 - 6 Dec 2022 - Allow processing new tasks first. Thanks @madrang

ui/index.html

@@ -47,7 +47,7 @@
 
     <div id="tab-content-wrapper">
         <div id="tab-content-main" class="tab-content active flex-container">
-        <div id="editor" class="scrollbar-editor">
+        <div id="editor">
             <div id="editor-inputs">
                 <div id="editor-inputs-prompt" class="row">
                     <label for="prompt"><b>Enter Prompt</b></label> <small>or</small> <button id="promptsFromFileBtn">Load from a file</button>
@@ -135,12 +135,6 @@
                         </select>
                         <a href="https://github.com/cmdr2/stable-diffusion-ui/wiki/VAE-Variational-Auto-Encoder" target="_blank"><i class="fa-solid fa-circle-question help-btn"><span class="simple-tooltip top-left">Click to learn more about VAEs</span></i></a>
                     </td></tr>
-                    <tr class="pl-5"><td><label for="hypernetwork_model">Hypernetwork:</i></label></td><td>
-                        <select id="hypernetwork_model" name="hypernetwork_model">
-                            <!-- <option value="" selected>None</option> -->
-                        </select>
-                    </td></tr>
-                    <tr id="hypernetwork_strength_container" class="pl-5"><td><label for="hypernetwork_strength_slider">Hypernetwork Strength:</label></td><td> <input id="hypernetwork_strength_slider" name="hypernetwork_strength_slider" class="editor-slider" value="100" type="range" min="0" max="100"> <input id="hypernetwork_strength" name="hypernetwork_strength" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"><br/></td></tr></span>
                     <tr id="samplerSelection" class="pl-5"><td><label for="sampler">Sampler:</label></td><td>
                         <select id="sampler" name="sampler">
                             <option value="plms">plms</option>
@@ -202,7 +196,16 @@
                     </td></tr>
                     <tr class="pl-5"><td><label for="num_inference_steps">Inference Steps:</label></td><td> <input id="num_inference_steps" name="num_inference_steps" size="4" value="25" onkeypress="preventNonNumericalInput(event)"></td></tr>
                     <tr class="pl-5"><td><label for="guidance_scale_slider">Guidance Scale:</label></td><td> <input id="guidance_scale_slider" name="guidance_scale_slider" class="editor-slider" value="75" type="range" min="10" max="500"> <input id="guidance_scale" name="guidance_scale" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"></td></tr>
-                    <tr id="prompt_strength_container" class="pl-5"><td><label for="prompt_strength_slider">Prompt Strength:</label></td><td> <input id="prompt_strength_slider" name="prompt_strength_slider" class="editor-slider" value="80" type="range" min="0" max="99"> <input id="prompt_strength" name="prompt_strength" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"><br/></td></tr></span>
+                    <tr id="prompt_strength_container" class="pl-5"><td><label for="prompt_strength_slider">Prompt Strength:</label></td><td> <input id="prompt_strength_slider" name="prompt_strength_slider" class="editor-slider" value="80" type="range" min="0" max="99"> <input id="prompt_strength" name="prompt_strength" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"><br/></td></tr>
+                    <tr class="pl-5"><td><label for="hypernetwork_model">Hypernetwork:</i></label></td><td>
+                        <select id="hypernetwork_model" name="hypernetwork_model">
+                            <!-- <option value="" selected>None</option> -->
+                        </select>
+                    </td></tr>
+                    <tr id="hypernetwork_strength_container" class="pl-5">
+                        <td><label for="hypernetwork_strength_slider">Hypernetwork Strength:</label></td>
+                        <td> <input id="hypernetwork_strength_slider" name="hypernetwork_strength_slider" class="editor-slider" value="100" type="range" min="0" max="100"> <input id="hypernetwork_strength" name="hypernetwork_strength" size="4" pattern="^[0-9\.]+$" onkeypress="preventNonNumericalInput(event)"><br/></td>
+                    </tr>
                     <tr class="pl-5"><td><label for="output_format">Output Format:</label></td><td>
                         <select id="output_format" name="output_format">
                             <option value="jpeg" selected>jpeg</option>

ui/media/css/image-editor.css

@@ -136,6 +136,10 @@
 	background: var(--background-color3);
 }
 
+.editor-controls-right .image-editor-button {
+	margin-bottom: 4px;
+}
+
 #init_image_button_inpaint .input-toggle {
 	position: absolute;
 	left: 16px;

ui/media/css/main.css

@@ -139,7 +139,7 @@ code {
     padding: 16px;
     display: flex;
     flex-direction: column;
-    flex: 0 0 370pt;
+    flex: 0 0 380pt;
 }
 #editor label {
     font-weight: normal;
@@ -797,6 +797,9 @@ input::file-selector-button {
 }
 
 @media (min-width: 700px) {
+    /* #editor {
+        max-width: 480px;
+    }*/
     .float-container {
         padding: 20px;
     }
@@ -805,53 +808,6 @@ input::file-selector-button {
         float: left;
         padding: 20px;
     }
-    #editor {
-        position:  fixed;
-        overflow-y: auto;
-        top: 0;
-        bottom: 0;
-        overflow-x: hidden;
-        padding-right: 8px;
-        z-index: 1;
-        width: 374pt;
-    }
-
-    #preview {
-        padding-left: 380pt;
-        margin-right: 10pt;
-        left: 0;
-        right:0;
-        min-height: 80vh;
-    }
-
-    #preview-pane {
-        display: none;
-        overflow-y: auto;
-        margin-top: 8pt;
-        left: 0;
-        right:0;
-    }
-
-    #preview-tools {
-        background: var(--background-color1);
-        position: sticky;
-        top: 0;
-        transition: 0.25s;
-        z-index: 1;
-        padding-top:10px;
-        padding-bottom: 10px;
-        -webkit-mask-image: linear-gradient(to bottom, black 0%, black 90%, transparent 100%);
-        mask-image: linear-gradient(to bottom, black 0%, black 50%, transparent 100%);
-    }
-
-    #editor-modifiers {
-        overflow-y: initial;
-        overflow-x: initial;
-    }
-    .image_preview_container {
-        padding: 6px;
-        width: 454px;
-    }
 }
 
 .help-btn {
@@ -1126,7 +1082,19 @@ div.task-initimg > img {
 }
 div.task-fs-initimage {
     display:  none;
+#    position: absolute;
+}
+div.task-initimg:hover div.task-fs-initimage {
+    display: block;
     position: absolute;
+    z-index: 9999;
+    box-shadow: 0 0 30px #000;
+    margin-top:-64px;
+}
+div.top-right {
+    position: absolute;
+    top: 8px;
+    right: 8px;
 }
 
 button#save-system-settings-btn {

ui/media/js/dnd.js

@@ -194,6 +194,28 @@ const TASK_MAPPING = {
         readUI: () => vaeModelField.value,
         parse: (val) => val
     },
+    use_hypernetwork_model: { name: 'Hypernetwork model',
+        setUI: (use_hypernetwork_model) => {
+            const oldVal = hypernetworkModelField.value
+
+            if (use_hypernetwork_model !== '') {
+                use_hypernetwork_model = getModelPath(use_hypernetwork_model, ['.pt'])
+                use_hypernetwork_model = use_hypernetwork_model !== '' ? use_hypernetwork_model : oldVal
+            }
+            hypernetworkModelField.value = use_hypernetwork_model
+            hypernetworkModelField.dispatchEvent(new Event('change'))
+        },
+        readUI: () => hypernetworkModelField.value,
+        parse: (val) => val
+    },
+    hypernetwork_strength: { name: 'Hypernetwork Strength',
+        setUI: (hypernetwork_strength) => {
+            hypernetworkStrengthField.value = hypernetwork_strength
+            updateHypernetworkStrengthSlider()
+        },
+        readUI: () => parseFloat(hypernetworkStrengthField.value),
+        parse: (val) => parseFloat(val)
+    },
 
     num_outputs: { name: 'Parallel Images',
         setUI: (num_outputs) => {
@@ -338,7 +360,9 @@ const TASK_TEXT_MAPPING = {
     use_upscale: 'Use Upscaling',
     sampler: 'Sampler',
     negative_prompt: 'Negative Prompt',
-    use_stable_diffusion_model: 'Stable Diffusion model'
+    use_stable_diffusion_model: 'Stable Diffusion model',
+    use_hypernetwork_model: 'Hypernetwork model',
+    hypernetwork_strength: 'Hypernetwork Strength'
 }
 const afterPromptRe = /^\s*Width\s*:\s*\d+\s*(?:\r\n|\r|\n)+\s*Height\s*:\s*\d+\s*(\r\n|\r|\n)+Seed\s*:\s*\d+\s*$/igm
 function parseTaskFromText(str) {

ui/media/js/engine.js

@@ -1107,7 +1107,7 @@
             }
             // Calling idle could result in task being added to queue.
             if (task_queue.size <= 0 && concurrent_generators.size <= 0) {
-                return idleEventPromise.then(() => asyncDelay(IDLE_COOLDOWN))
+                return asyncDelay(IDLE_COOLDOWN).then(() => idleEventPromise)
             }
         }
         if (task_queue.size < serverCapacity) {

ui/media/js/image-editor.js

@@ -105,7 +105,26 @@ const IMAGE_EDITOR_ACTIONS = [
 		icon: "fa-solid fa-xmark",
 		handler: (editor) => {
 			editor.ctx_current.clearRect(0, 0, editor.width, editor.height)
-		}
+		},
+		trackHistory: true
+	},
+	{
+		id: "undo",
+		name: "Undo",
+		icon: "fa-solid fa-rotate-left",
+		handler: (editor) => {
+			editor.history.undo()
+		},
+		trackHistory: false
+	},
+	{
+		id: "redo",
+		name: "Redo",
+		icon: "fa-solid fa-rotate-right",
+		handler: (editor) => {
+			editor.history.redo()
+		},
+		trackHistory: false
 	}
 ]
 
@@ -436,13 +455,14 @@ class ImageEditor {
 			return
 		}
 
-		var max_size = Math.min(parseInt(window.innerWidth * 0.9), width, 768)
 		if (width > height) {
+		        var max_size = Math.min(parseInt(window.innerWidth * 0.9), width, 768)
 			var multiplier = max_size / width
 			width = (multiplier * width).toFixed()
 			height = (multiplier * height).toFixed()
 		}
 		else {
+		        var max_size = Math.min(parseInt(window.innerHeight * 0.9), height, 768)
 			var multiplier = max_size / height
 			width = (multiplier * width).toFixed()
 			height = (multiplier * height).toFixed()
@@ -525,7 +545,9 @@ class ImageEditor {
 	}
 	runAction(action_id) {
 		var action = IMAGE_EDITOR_ACTIONS.find(a => a.id == action_id)
+		if (action.trackHistory) {
 			this.history.pushAction(action_id)
+		}
 		action.handler(this)
 	}
 	setBrush(layer = null, options = null) {

ui/media/js/main.js

@@ -62,29 +62,6 @@ let maskSetting = document.querySelector('#enable_mask')
 
 const processOrder = document.querySelector('#process_order_toggle')
 
-const editorContainer = document.querySelector('#editor')
-window.addEventListener("scroll", updatePreviewSize)
-let lastScrollTop = 0
-updatePreviewSize()
-
-// update preview pane size and position
-function updatePreviewSize() {
-    const scrollTop = window.pageYOffset || document.documentElement.scrollTop
-    if (scrollTop > lastScrollTop) {
-        previewTools.style.top = -previewTools.offsetHeight + 'px'
-    }
-    else if (scrollTop < lastScrollTop) {
-        const elem = preview.getElementsByClassName('img-batch')[0]
-        if (elem !== undefined && Math.round(window.scrollY) !== Math.round(elem.closest(".imageTaskContainer").offsetTop)) {
-            previewTools.style.top = '0'
-        }
-    }
-    lastScrollTop = scrollTop
-
-    $('#editor').css('top', Math.max(-window.pageYOffset + $("#tab-container").offset().top + $('#tab-container').outerHeight(true), 0) + 'px')
-    $('#editor').css('bottom', Math.max($(window).height() - ($("#footer .line-separator").offset().top - $(document).scrollTop()), 0) + 'px')
-};
-
 let imagePreview = document.querySelector("#preview")
 imagePreview.addEventListener('drop', function(ev) {
     const data = ev.dataTransfer?.getData("text/plain");
@@ -293,9 +270,9 @@ function showImages(reqBody, res, outputContainer, livePreview) {
         imageElem.setAttribute('data-steps', imageInferenceSteps)
         imageElem.setAttribute('data-guidance', imageGuidanceScale)
 
+
         const imageInfo = imageItemElem.querySelector('.imgItemInfo')
         imageInfo.style.visibility = (livePreview ? 'hidden' : 'visible')
-        updatePreviewSize()
 
         if ('seed' in result && !imageElem.hasAttribute('data-seed')) {
             const req = Object.assign({}, reqBody, {
@@ -437,7 +414,11 @@ function getUncompletedTaskEntries() {
         document.querySelectorAll('#preview .imageTaskContainer .taskStatusLabel')
         ).filter((taskLabel) => taskLabel.style.display !== 'none'
         ).map(function(taskLabel) {
-            return taskLabel.closest('.imageTaskContainer')
+            let imageTaskContainer = taskLabel.parentNode
+            while(!imageTaskContainer.classList.contains('imageTaskContainer') && imageTaskContainer.parentNode) {
+                imageTaskContainer = imageTaskContainer.parentNode
+            }
+            return imageTaskContainer
         })
     if (!processOrder.checked) {
         taskEntries.reverse()
@@ -458,10 +439,10 @@ function makeImage() {
         alert('The "Inference Steps" field must not be empty.')
         return
     }
-    if (numOutputsTotalField.value == '') {
+    if (numOutputsTotalField.value == '' || numOutputsTotalField.value == 0) {
         numOutputsTotalField.value = 1
     }
-    if (numOutputsParallelField.value == '') {
+    if (numOutputsParallelField.value == '' || numOutputsParallelField.value == 0) {
         numOutputsParallelField.value = 1
     }
     if (guidanceScaleField.value == '') {
@@ -653,7 +634,9 @@ function onTaskCompleted(task, reqBody, instance, outputContainer, stepUpdate) {
     time /= 1000
 
     if (task.batchesDone == task.batchCount) {
+	if (!task.outputMsg.innerText.toLowerCase().includes('error')) {
             task.outputMsg.innerText = `Processed ${task.numOutputsTotal} images in ${time} seconds`
+	}
         task.progressBar.style.height = "0px"
         task.progressBar.style.border = "0px solid var(--background-color3)"
         task.progressBar.classList.remove("active")
@@ -767,23 +750,11 @@ async function onTaskStart(task) {
 /* Hover effect for the init image in the task list */
 function createInitImageHover(taskEntry) {
     var $tooltip = $( taskEntry.querySelector('.task-fs-initimage') )
-    $( taskEntry.querySelector('div.task-initimg > img') ).on('mouseenter', function() {
+    var img = document.createElement('img')
-        var img = this,
+    img.src = taskEntry.querySelector('div.task-initimg > img').src
-           $img = $(img),
+    $tooltip.append(img)
-           offset = $img.offset();
+    $tooltip.append(`<div class="top-right"><button>Use as Input</button></div>`)
-    
+    $tooltip.find('button').on('click', (e) => { onUseAsInputClick(null,img) } )
-       $tooltip
-       .css({
-           'top': offset.top,
-           'left': offset.left,
-           'z-index': 99999,
-           'display': 'block'
-       })
-       .append($img.clone().css({width:"", height:""}));
-    })
-    $tooltip.on('mouseleave', function() {
-       $tooltip.empty().addClass('hidden');
-    });
 }
 
 function createTask(task) {
@@ -846,6 +817,8 @@ function createTask(task) {
     task['stopTask'] = taskEntry.querySelector('.stopTask')
 
     task['stopTask'].addEventListener('click', (e) => {
+        e.stopPropagation()
+
         let question = (task['isProcessing'] ? "Stop this task?" : "Remove this task?")
         shiftOrConfirm(e, question, async function(e) {
             if (task.batchesDone <= 0 || !task.isProcessing) {
@@ -1080,7 +1053,6 @@ function removeTask(taskToRemove) {
         previewTools.style.display = 'none'
         initialText.style.display = 'block'
     }
-    updatePreviewSize()
 }
 
 clearAllPreviewsBtn.addEventListener('click', (e) => { shiftOrConfirm(e, "Clear all the results and tasks in this window?", async function() {
@@ -1202,6 +1174,12 @@ hypernetworkStrengthSlider.addEventListener('input', updateHypernetworkStrength)
 hypernetworkStrengthField.addEventListener('input', updateHypernetworkStrengthSlider)
 updateHypernetworkStrength()
 
+function updateHypernetworkStrengthContainer() {
+    document.querySelector("#hypernetwork_strength_container").style.display = (hypernetworkModelField.value === "" ? 'none' : '')
+}
+hypernetworkModelField.addEventListener('change', updateHypernetworkStrengthContainer)
+updateHypernetworkStrengthContainer()
+
 /********************* JPEG Quality **********************/
 function updateOutputQuality() {
     outputQualityField.value =  0 | outputQualitySlider.value
@@ -1276,6 +1254,10 @@ async function getModels() {
         vaeOptions.forEach(createModelOptions(vaeModelField, selectedVaeModel))
         hypernetworkOptions.forEach(createModelOptions(hypernetworkModelField, selectedHypernetworkModel))
 
+        stableDiffusionModelField.dispatchEvent(new Event('change'))
+        vaeModelField.dispatchEvent(new Event('change'))
+        hypernetworkModelField.dispatchEvent(new Event('change'))
+
         // TODO: set default for model here too
         SETTINGS[sd_model_setting_key].default = stableDiffusionOptions[0]
         if (getSetting(sd_model_setting_key) == '' || SETTINGS[sd_model_setting_key].value == '') {
@@ -1393,6 +1375,7 @@ function selectTab(tab_id) {
         tabInfo.tab.classList.toggle("active")
         tabInfo.content.classList.toggle("active")
     }
+    document.dispatchEvent(new CustomEvent('tabClick', { detail: tabInfo }))
 }
 function linkTabContents(tab) {
     var name = tab.id.replace("tab-", "")

ui/media/js/utils.js

@@ -45,6 +45,7 @@ function toggleCollapsible(element) {
             handle.innerHTML = '➖' // minus
         }
     }
+    document.dispatchEvent(new CustomEvent('collapsibleClick', { detail: collapsibleHeader }))
     
     if (COLLAPSIBLES_INITIALIZED && COLLAPSIBLE_PANELS.includes(element)) {
         saveCollapsibles()

ui/sd_internal/runtime.py

@@ -822,7 +822,7 @@ def _txt2img(opt_W, opt_H, opt_n_samples, opt_ddim_steps, opt_scale, start_code,
         move_to_cpu(thread_data.modelCS)
 
     if thread_data.test_sd2 and sampler_name not in ('plms', 'ddim', 'dpm2'):
-        raise Exception('Only plms and ddim samplers are supported right now, in SD 2.0')
+        raise Exception('Only plms, ddim and dpm2 samplers are supported right now, in SD 2.0')
 
 
     # samples, _ = sampler.sample(S=opt.steps,

ui/sd_internal/task_manager.py

@@ -372,6 +372,8 @@ def thread_render(device):
             session_cache.keep(task.request.session_id, TASK_TTL)
         except Exception as e:
             task.error = e
+            task.response = {"status": 'failed', "detail": str(task.error)}
+            task.buffer_queue.put(json.dumps(task.response))
             print(traceback.format_exc())
             continue
         finally: