How to use the Stable Diffusion web UI checkpoint model
About the Stable Diffusion Model
Stable Diffusion models, or checkpoint models, are data that have been pre-trained to produce a particular style of image.
The type of image the model generates depends on the training images. If the training data does not contain any cat images, the model will not produce any cat images. Similarly, if the model is trained with only cat images, it will produce only cat images.
To find a model, look for a 🔗Cibitai or 🔗Hugging Face or other model of your choice.
Civitai
Cibitai is a service started in 2022 for the development of AI art and user interaction, and is a platform for downloading models available on Stable Diffusion. In addition to model downloads, Cibitai offers other services such as AI art, article submission, on-site image generation, and Lora training. The home page is also easy to view with thumbnail listings, and we recommend this site for beginners.
Cibitai-How to download models
We will refer to Aitasai’s Dark Sushi Mix model for explanation. *This model will be explained in detail later.
Hugging Face (Hugging Face Hub)
Hugging Face (Hugging Face Hub) is a Git-based hosting service started in 2022 by Hugging Face, Inc. In addition to model downloads, text, image, and music datasets are shared. The website is more specialized than Civitai and is not suitable for beginners.
Hugging Face-How to Download Models
Here 🔗Vsukiyaki’s 🔗ShiratakiMix is used as an example.
.ckpt and .safetensors files, download the .safetensors file.
There are two types of model files: .ckpt and .safetensors. Originally, model files were shared in .ckpt format, but .ckpt files have the problem that malicious code can be embedded in them. Hugging Face has developed safer and faster .safetensors. Not all .ckpt files are dangerous, but if you have no particular reason to use .ckpt, use .safetensors.
Checkpoint Model Introduction
For reference, this is the checkpoint model used to generate the top image for this article.
- Creator: 🔗Aitasai
- Download: 1.99 GB
- Upload: 2023/6/11
- File Format: SafeTensor
- Base Model: SD 1.5
- Recommended VAE: 🔗vae-ft-mse-840000-ema *This article does not explain how to use VAE.
- Hires. Fix recommended
The following articles also provide recommended checkpoints.
![Featured image of [ Stable Diffusion web UI ] Introduction of SD1.5 Recommended Checkpoint Model](https://dca.data-hub-center.com/content/uploads/2024/05/eyecatch-sd15-recommended-checkpoint-models-en.jpg)
![Featured image of [ Stable Diffusion web UI ] Introduction of SDXL1.0 Recommended Checkpoint Models](https://dca.data-hub-center.com/content/uploads/2024/06/eye_catch_en_sdxl-recommended-checkpoint-models.jpg)
How to Install the Checkpoint Model
To use the checkpoint model with Stable Diffusion web UI, move the darkSushiMixMix_225D.safetensors file you downloaded into the \stable-diffusion-webui\models\Stable-diffusion Move the file to the diffusion folder.
Return to the browser and press the 🔄 button next to the checkpoint model selection tab in the upper left corner.
When the update is complete, the checkpoint model that was just moved to the folder is displayed from the tab, so select it and load it.
How to use the checkpoint model
Let’s paste the following prompt as a test.
(ultra art illustrated style, masterpiece:1.3), ultra detailed, dutch_angle,
1girl, beautiful face,
(medival, mage:1.1), hill_side, blue_sky,horizonPaste the following prompts into the negative prompt as well.
(worst quality:2.0), (low quality:2.0), (normal quality:2.0), lowresEnter the settings as follows and press the “Generate” button.
- Sampling method: DPM++ SDE
- Schedule type: Karras
- Sampling steps: 35
- Width: 768
- Height: 512
- CFG Scale: 5
- Seed: 2454522269
- Batch count: 1
- Batch size: 1
We think a fantasy-like girl has been generated, but the resolution is 768×512, so it looks a little rough, doesn’t it? Let’s use “Hires. Fix” to add more details.
Leave the “Hires. Fix” checkbox unchecked and set the settings as follows
- Upscaler：R-ESRGAN 4x+
- Hires steps：15
- Denoising strength：0.3
- Upscale by：2
- Resize width to：0
- Resize height to：0
When you press the “Generate” button and your favorite image comes up, press the ✨ button on the far right side of the line below the preview image to apply the Hires. Fix to that image.
About the base model
There are two primary versions of the “base model,” SD1.5 and SDXL. The features of each are described below.
SD1.5
- Training-based resolution: 512 x 512 dot
- Text Encoder: Open AI CLIP ViT-L/14
- Key Features: SD1.5 generates images at a relatively low resolution. While the prompts are intuitive and easy to handle, the detail of the generated images is limited.
SDXL
- Training-based resolution: 1,024 x 1,024 dot
- Text Encoder: OpenClip model (ViT-G/14) & OpenAI proprietary CLIP ViT-L.
-
Key Features:
- Highly prompt comprehension: The addition of the OpenClip model to the text encoder further improves prompt comprehension.
- High processing capacity: It can handle larger and more complex image generation tasks.
- High quality images: The level of realism and detail has been improved and can produce very high resolution images.
- Speed: It works faster than previous models and reduces the time it takes to generate images without compromising image quality.
The difference between Pruned, Full and EMA-only models
- Pruned Model: Lightweight models that are pruned after model training is complete and redundant parts are removed. Used in resource-limited environments or when fast generation is required.
- Full Model: Model before lightweighting. Excellent accuracy because it has all the information learned.
- EMA-only Model: Similar to the Pruned Model, but with noise smoothing applied to the final few steps during training. It is able to generate images with stable accuracy while reducing data weight.
About Fp16 and Fp32
The model data is labeled Fp16 or Fp32. Fp” stands for Floating point. Fp16 is used in resource-limited environments and in applications that require fast processing, since it reduces memory usage and increases calculation speed. Fp32, on the other hand, is suitable when higher precision is required or when working with large data sets.
Checkpoint model data can be a very large file, and trying out different model data will put pressure on your storage. If you have no particular preference, download Pluned’s Fp16.
Conclusion
This article details the use of the checkpoint model in Stable Diffusion’s Web UI; using the Stable Diffusion model allows you to effectively control the style of the generated images. In addition, the model is updated regularly, with new versions being added, so you can always take advantage of the latest technology. In addition to Checkpoint, there are many other features such as Lola, Textual inversions (embedding), VAE, and Hypernetworks, which we will discuss in more detail at another time.
